-- Logs begin at Wed 2022-08-31 03:54:15 UTC, end at Wed 2022-08-31 04:53:28 UTC. -- Aug 31 04:20:40.742524 dv-cinder-855307-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 04:20:40.750709 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 04:20:40 2022] *** Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: nodename: dv-cinder-855307-1-iscsi Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: machine: x86_64 Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: clock source: unix Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: pcre jit disabled Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: detected number of CPU cores: 4 Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: current working directory: / Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: detected binary path: /usr/bin/uwsgi-core Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: your processes number limit is 47680 Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: your memory page size is 4096 bytes Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: detected max file descriptor number: 2048 Aug 31 04:20:40.752804 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: lock engine: pthread robust mutexes Aug 31 04:20:40.753566 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: thunder lock: enabled Aug 31 04:20:40.753566 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 04:20:40.753566 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 04:20:40.793909 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: Python main interpreter initialized at 0x55837803b9c0 Aug 31 04:20:40.793909 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: python threads support enabled Aug 31 04:20:40.793909 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: your server socket listen backlog is limited to 100 connections Aug 31 04:20:40.793909 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: your mercy for graceful operations on workers is 90 seconds Aug 31 04:20:40.794344 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 04:20:40.794344 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: *** Operational MODE: preforking *** Aug 31 04:20:40.794645 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: *** uWSGI is running in multiple interpreter mode *** Aug 31 04:20:40.794645 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: spawned uWSGI master process (pid: 81556) Aug 31 04:20:40.794878 dv-cinder-855307-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 04:20:40.799520 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: spawned uWSGI worker 1 (pid: 81569, cores: 1) Aug 31 04:20:40.799520 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: spawned uWSGI worker 2 (pid: 81570, cores: 1) Aug 31 04:20:40.799520 dv-cinder-855307-1-iscsi devstack@keystone.service[81556]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 04:20:45.076571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 04:20:45.076571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Aug 31 04:20:45.077261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 04:20:45.077261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: from cryptography.utils import int_from_bytes Aug 31 04:20:45.077667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 04:20:45.077667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Aug 31 04:20:45.077773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 04:20:45.077773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: from cryptography.utils import int_from_bytes Aug 31 04:20:45.126125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 04:20:45.126330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 04:20:45.126454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] command line args: [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 04:20:45.126749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 04:20:45.126926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 04:20:45.127021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ================================================================================ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 04:20:45.127134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 04:20:45.127226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] admin_token = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.127331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] command line args: [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 04:20:45.127491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] config_dir = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.127610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 04:20:45.127696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.127766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ================================================================================ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 04:20:45.127831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] config_source = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.127964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] debug = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_dir = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128742 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_source = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] debug = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.128948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.129217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.129315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.129413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.129552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.129773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.129923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.129923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_config_append = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_dir = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_config_append = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_file = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_dir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.130978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.131991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.132951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133115 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.133997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.134227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.134358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.134436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] publish_errors = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.134580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.134689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.134843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.134929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] publish_errors = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.135012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.135270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.135398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.135486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.135564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.135657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.135737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] standard_threads = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.135903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] standard_threads = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_journal = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_json = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_stderr = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.136950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_syslog = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.137040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.137112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_journal = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.137180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.137291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_json = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.137415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.137500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_stderr = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.137648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_syslog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.137766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.137843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 04:20:45.137927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.138117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.138238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.138315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.138399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.138584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.138748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.138831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.138913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.139073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.139201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.139286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.139368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.139552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.139684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.139776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.139850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.140943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141393 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.141952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.142105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.142265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.142340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.142409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.142593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.142673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.142846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.142926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.143100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.143179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.143327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.143414 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.143553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.143634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.143781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.143867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.144008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.144095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.144251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.144331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.144490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.144575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.144733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.144809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.145044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.145124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.145350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.145434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.145584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.145717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.145805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.145931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.146033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.146179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.146282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.146409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.146529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.146645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.146798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.146915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.147981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.148985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.149134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.149208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.149372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.149448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.149601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.149676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.149837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.149910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.150071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.150138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.150311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.150375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.150537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.150607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.150791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.150861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.151950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.152976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.153119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.153203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.153354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.153432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.153583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.153662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.153821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.153896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.154057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.154133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.154304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.154390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.154538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.154614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.154801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.154881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155504 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.155976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.156991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.157146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.157233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.157378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.157454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.157609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.157684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.157845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.157927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.158084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.158160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.158321 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.158396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.158557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.158639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.158833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.158916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.159081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.159158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.159327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.159481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.159556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.159624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.159790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.159866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.160986 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.161966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.162125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.162200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.162377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.162454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.162615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.162691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.162878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.external = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.162968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.163109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.163214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.163380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.163461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.163613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.external = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.163690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.163840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.password = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.163916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.164068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.token = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.164148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.164307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.164385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.164539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.164615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.password = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.164783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.164861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.token = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165492 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.165954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.166027 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.166198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.166275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.166429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.166504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.166670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.166807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.166937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167732 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.167978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.168133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.168208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.168377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.168454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.168630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.168706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.168868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.168946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.169119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.169198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.169362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.169437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.169617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.169701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.169853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.169932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.170088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.170166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.170330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.170407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.170573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.170652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.170827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.170908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.171073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.171149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.171307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.171385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.171537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.171613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.171781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.171865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172492 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.172972 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.173995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.174137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.174215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.174375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.174450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.174627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.174762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.174865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.174942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.175101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.175179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.175330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.175466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.175545 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.175722 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.175798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.176993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.177071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.177240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.177318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.177476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.177552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.177708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.177793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.177944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.178993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.179147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.179226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.179379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.179454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.179618 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.179694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.179860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.179939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.180093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.180171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.180327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.180403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.180571 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.180648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.180800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.180879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.181966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.182043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.182196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.182272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.182429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.182513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.182665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.182758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.182932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183249 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183715 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.183965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.184109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.184185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.184340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.184423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.184571 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.184656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.184800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.184878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185815 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.185977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.186062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.186219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.186295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.186452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.186529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.186678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.186798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.186934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.187936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.188099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.188185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.188335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.188413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.188565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.188643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.189169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.189262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.189396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.189474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.189640 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.189719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.189875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.189961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.190109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.190187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.190336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.190412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.190567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.190651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.190829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.190910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.191069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.191146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.191306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.191395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.191600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.191682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.191834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.191995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.192072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.192255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.192351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.192490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.192568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.192737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.192813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.192968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.193085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.193216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.193327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.193457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.193558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.193691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.193792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.193914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.194031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.194160 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.194280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.194400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.194527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.194632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.194784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.194891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.195995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.196073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.196233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.196310 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.196465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] role.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.196621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.196696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] role.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.196767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.196926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.197963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.198136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.198212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.198381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.198456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.198624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.198702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.198893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.198970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.199123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.199199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.199369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.199448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.199612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.199689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.199855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.199931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.200083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.200161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.200330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.200472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.200575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.200708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.200822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.200956 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.201067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.201203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.201446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.201527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.201689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.201767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.201919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.201994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.202171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.202256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.202415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.202528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.202652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.202755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.202906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.203926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.204088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.204165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.204318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.204403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.204549 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.204626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.204776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.204853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.205953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.206029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.206193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.206269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.206423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.206507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.206662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.206757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.206926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.207009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.207237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.207333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.207475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.207554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.207711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.207788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.207945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.208989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.209146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.209224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.209404 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.209481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.209636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.209723 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.209877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.209955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.210116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.210195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.210353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.210431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.210587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.210666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.210844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.210924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.211080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.211157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.211313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.211392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.211572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.211657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.211790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.211867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212027 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.212971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.213048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.213199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.213276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.213435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.213514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.213665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.213742 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.213833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 04:20:45.213911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.214126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.214365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.214599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 04:20:45.214806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 04:20:45.243052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.243279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.243506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.243617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.243852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.244081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.251862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.252514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.280563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.280712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.280850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.280930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.335457 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.335625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.383782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.384039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.384990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.385227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.542654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.542892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.543105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.543227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.543328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.543408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.543471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.543577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.543665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.543796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.544040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.544262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.567197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.567331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.567485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.567628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.567712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.567833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.567915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.567978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.568048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.568189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.568311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.568380 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.568440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.568612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.579389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.579902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.592492 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.592643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.592720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.592849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.592932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.592995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.593099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.593187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.593341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.593433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.593496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.593640 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.593730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.593803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 04:20:45.610621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.610858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 04:20:45.610979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.611191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 04:20:45.611311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.611403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 04:20:45.611506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.611663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.611786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 04:20:45.611939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.612045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.612206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.612438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.612561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.612704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.612827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.622290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.622412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.622551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.622682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.622870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.623016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.623128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.623221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.630225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.630423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.633130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.633395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.633518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.633677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.633799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.633959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.634068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.634229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.634352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.634464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.634553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.634744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.634835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.635046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.635174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.635297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.635404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.635570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.635656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.635848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.635974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.636219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.658462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.659170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.659950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.660229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.660732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.661020 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.664939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.665223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.665501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.665689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.665994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.666277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.672062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.672328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.672707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.673000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.676346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.676636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.676906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.677174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.677266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.677466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.677592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.677745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.678035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.678298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.678583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.678883 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.679159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.679525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.679810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.680097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.696042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.696309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.697016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.697303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.700205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.701202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.703427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.703692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.704493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.704792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.708083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.709107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.711215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.712283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.714300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.714592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.714905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.715189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.715449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.715570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.715814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.715936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.716103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.716370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.716687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.716959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.731677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.731967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.732233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.732506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.733109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.733401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.733673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.733944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.739407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.741064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.741234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.742622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.742791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.742885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.744275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.744539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.745547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.745816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.746114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.746376 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.746694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.747238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.747526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.747912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.748200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.748501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.758021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.758296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.758576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.758881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.759164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.759447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.759731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.760043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.760331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.760725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.761012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.761280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.761560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.761859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.775191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.777620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.778234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.780603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.781143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.782585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.783518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.784086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.784367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.784659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.784962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.786678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.786987 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.787298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.791304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.792784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.793730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.795265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.795702 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.795989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.796279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.796559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.798274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.798579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.798896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.799172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.804647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.804939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.805216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.807329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.807625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.807910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.812119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.812406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.812684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.812960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.813228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.813496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.814123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.814422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.814697 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.815007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.815269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 04:20:45.815395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.815495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.815586 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.815747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.815875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.816018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.816101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.816316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.816655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.816936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.817220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.817509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.817793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.818070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.818336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.818607 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.818916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 04:20:45.850314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55837803b9c0 pid: 81569 (default app) Aug 31 04:20:45.853453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55837803b9c0 pid: 81570 (default app) Aug 31 04:20:45.853841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6f29-93b6-4ae8-9529-3c3aa22ad40e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:45.854228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6f29-93b6-4ae8-9529-3c3aa22ad40e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:45.854537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6f29-93b6-4ae8-9529-3c3aa22ad40e None None] PATH_INFO: `/v3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:45.858745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Wed Aug 31 04:20:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:20:52.438258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9e85f-1dfc-4d57-b38a-26eefe7d0434 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:52.438821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9e85f-1dfc-4d57-b38a-26eefe7d0434 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:52.438821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9e85f-1dfc-4d57-b38a-26eefe7d0434 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:52.445415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:20:52 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:20:52.451481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:52.451962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:52.452101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:52.521969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: INFO dbcounter [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] Registered counter for database keystone Aug 31 04:20:52.543347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG oslo_db.sqlalchemy.engines [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81569) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 04:20:52.554358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writer thread running {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 04:20:52.736071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG passlib.handlers.bcrypt [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81569) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 04:20:52.736358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG passlib.handlers.bcrypt [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81569) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 04:20:52.795292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:20:52.796878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:20:52.931301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:20:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 04:20:53.405000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9e85f-1dfc-4d57-b38a-26eefe7d0434 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:53.484412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: INFO dbcounter [None req-e8e9e85f-1dfc-4d57-b38a-26eefe7d0434 None None] Registered counter for database keystone Aug 31 04:20:53.509862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG oslo_db.sqlalchemy.engines [None req-e8e9e85f-1dfc-4d57-b38a-26eefe7d0434 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81570) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 04:20:53.519258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writer thread running {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 04:20:53.637325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9e85f-1dfc-4d57-b38a-26eefe7d0434 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:53.641486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8aa59-f9fa-48cb-ad6a-2f2f53f2e05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:53.642120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8aa59-f9fa-48cb-ad6a-2f2f53f2e05f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:53.642395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8aa59-f9fa-48cb-ad6a-2f2f53f2e05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:53.642659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8aa59-f9fa-48cb-ad6a-2f2f53f2e05f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:53.651226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8aa59-f9fa-48cb-ad6a-2f2f53f2e05f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:53.704199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d0c8aa59-f9fa-48cb-ad6a-2f2f53f2e05f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 04:20:53.704470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8aa59-f9fa-48cb-ad6a-2f2f53f2e05f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:53.715281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Aug 31 04:20:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:20:53.721268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:53.735898 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ba22a-f254-444c-b672-12d1d057a237 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:53.740522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6313430-3845-4f3b-b1b4-e76ce36a6222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:53.741483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6313430-3845-4f3b-b1b4-e76ce36a6222 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:53.741763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6313430-3845-4f3b-b1b4-e76ce36a6222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:53.742034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6313430-3845-4f3b-b1b4-e76ce36a6222 admin admin] PATH_INFO: `/v3/projects/529acf6d876d441682b1d4508345bc61` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:53.743575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6313430-3845-4f3b-b1b4-e76ce36a6222 admin admin] RBAC: Authorizing `identity:get_project(project_id=529acf6d876d441682b1d4508345bc61)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:53.797324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d6313430-3845-4f3b-b1b4-e76ce36a6222 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 04:20:53.797655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6313430-3845-4f3b-b1b4-e76ce36a6222 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:53.802326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Aug 31 04:20:53 2022] GET /identity/v3/projects/529acf6d876d441682b1d4508345bc61 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:20:55.705902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6360d7-a4d8-4e4c-9ccc-58e46abeed5a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:55.706868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6360d7-a4d8-4e4c-9ccc-58e46abeed5a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:55.706868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6360d7-a4d8-4e4c-9ccc-58e46abeed5a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:55.712402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:20:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:20:55.719100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48beb477-d973-4402-94e2-d731a0cbbbcb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:55.719537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48beb477-d973-4402-94e2-d731a0cbbbcb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:55.719630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48beb477-d973-4402-94e2-d731a0cbbbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:55.748332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-48beb477-d973-4402-94e2-d731a0cbbbcb None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:20:55.749874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-48beb477-d973-4402-94e2-d731a0cbbbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:20:55.760202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:20:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 04:20:56.024849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6360d7-a4d8-4e4c-9ccc-58e46abeed5a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:56.039322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6360d7-a4d8-4e4c-9ccc-58e46abeed5a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:56.044751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a140cc-ab5c-4708-bc3b-93fd37e4f7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:56.045352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a140cc-ab5c-4708-bc3b-93fd37e4f7ba admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:56.045629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a140cc-ab5c-4708-bc3b-93fd37e4f7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:56.045902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a140cc-ab5c-4708-bc3b-93fd37e4f7ba admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:56.054585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a140cc-ab5c-4708-bc3b-93fd37e4f7ba admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:56.056400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a140cc-ab5c-4708-bc3b-93fd37e4f7ba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:56.067062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Aug 31 04:20:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:20:56.073129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48beb477-d973-4402-94e2-d731a0cbbbcb None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:56.082625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48beb477-d973-4402-94e2-d731a0cbbbcb None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:56.088101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba161fb-4d9f-4c35-8471-29b898059ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:56.088741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba161fb-4d9f-4c35-8471-29b898059ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:56.089025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba161fb-4d9f-4c35-8471-29b898059ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:56.089305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba161fb-4d9f-4c35-8471-29b898059ac2 admin admin] PATH_INFO: `/v3/users/13d95fead9ed4b918730e2198000e5e5` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:56.090697 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba161fb-4d9f-4c35-8471-29b898059ac2 admin admin] RBAC: Authorizing `identity:get_user(user_id=13d95fead9ed4b918730e2198000e5e5)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:56.092596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba161fb-4d9f-4c35-8471-29b898059ac2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:56.094126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Wed Aug 31 04:20:56 2022] GET /identity/v3/users/13d95fead9ed4b918730e2198000e5e5 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:20:58.049150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f58df-f747-4728-b4fd-8c8ef6fbd16c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:58.049695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f58df-f747-4728-b4fd-8c8ef6fbd16c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:58.049695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f58df-f747-4728-b4fd-8c8ef6fbd16c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:58.051035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:20:58.056950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0b802-4b52-4aaa-a4e1-0cdfb979fb5f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:58.057364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0b802-4b52-4aaa-a4e1-0cdfb979fb5f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:58.057594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0b802-4b52-4aaa-a4e1-0cdfb979fb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:58.077019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0cf0b802-4b52-4aaa-a4e1-0cdfb979fb5f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:20:58.078208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0cf0b802-4b52-4aaa-a4e1-0cdfb979fb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:20:58.093049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b91b7-56f1-47df-8113-681f58fb7765 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:58.093330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b91b7-56f1-47df-8113-681f58fb7765 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:58.093599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b91b7-56f1-47df-8113-681f58fb7765 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:58.094967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:20:58.100002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:58.100281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:58.100547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:58.141871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81570) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 04:20:58.142270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81570) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 04:20:58.161147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:20:58.191313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:20:58.192884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:20:58.294354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:20:58.436759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0b802-4b52-4aaa-a4e1-0cdfb979fb5f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:58.462006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:58.545402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0b802-4b52-4aaa-a4e1-0cdfb979fb5f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:58.563049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:58.582282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee3571-f2dd-49d2-9ba9-58af25328cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:58.582961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee3571-f2dd-49d2-9ba9-58af25328cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:58.583269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee3571-f2dd-49d2-9ba9-58af25328cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:58.583547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee3571-f2dd-49d2-9ba9-58af25328cf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:58.595440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee3571-f2dd-49d2-9ba9-58af25328cf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-95ee3571-f2dd-49d2-9ba9-58af25328cf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:20:58.604902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:20:58.605970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:20:58.607074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:20:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:20:58.611926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee3571-f2dd-49d2-9ba9-58af25328cf9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:58.669988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930056ac-286b-4c11-9d7a-cbaee1eb33fe None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:58.673066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:58.695188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:58.708228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0057545-d701-4878-983d-ef3f454288cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:58.709101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0057545-d701-4878-983d-ef3f454288cd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:58.709482 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0057545-d701-4878-983d-ef3f454288cd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:58.709809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0057545-d701-4878-983d-ef3f454288cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:58.713963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee3571-f2dd-49d2-9ba9-58af25328cf9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:58.748510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:58.749147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:58.749422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:58.749692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:58.750389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:20:58.750683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:58.752119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:58.764211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:20:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:20:58.770531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:58.777372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0057545-d701-4878-983d-ef3f454288cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:58.778935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0057545-d701-4878-983d-ef3f454288cd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:58.868362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0887b12-0ef5-4455-8436-a90e7356c2ef None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:58.878565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:20:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 04:20:58.886110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0057545-d701-4878-983d-ef3f454288cd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:58.889167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:58.904725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa19424-05fe-4849-9c61-dd869e8f7f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:58.905379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa19424-05fe-4849-9c61-dd869e8f7f5f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:58.905653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa19424-05fe-4849-9c61-dd869e8f7f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:58.905929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa19424-05fe-4849-9c61-dd869e8f7f5f None admin] PATH_INFO: `/v3/users/13d95fead9ed4b918730e2198000e5e5` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:58.907129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa19424-05fe-4849-9c61-dd869e8f7f5f None admin] RBAC: Authorizing `identity:get_user(user_id=13d95fead9ed4b918730e2198000e5e5)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:58.908609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa19424-05fe-4849-9c61-dd869e8f7f5f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:58.910189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:20:58 2022] GET /identity/v3/users/13d95fead9ed4b918730e2198000e5e5 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:20:58.917626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa19424-05fe-4849-9c61-dd869e8f7f5f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:58.934659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:58.989279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0057545-d701-4878-983d-ef3f454288cd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:59.021554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa19424-05fe-4849-9c61-dd869e8f7f5f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:59.023605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:59.024511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:59.024785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:59.025047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:59.043698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:59.045178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:59.057570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30407de-4cdf-46ef-9f07-abcfc4fdfaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:59.058209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30407de-4cdf-46ef-9f07-abcfc4fdfaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:59.058489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30407de-4cdf-46ef-9f07-abcfc4fdfaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:59.058763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30407de-4cdf-46ef-9f07-abcfc4fdfaa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:59.060009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30407de-4cdf-46ef-9f07-abcfc4fdfaa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:59.061734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30407de-4cdf-46ef-9f07-abcfc4fdfaa9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:20:59.063340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:20:59.064838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:20:59.065623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:20:59.065623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:20:59.065623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:20:59.065785 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:20:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:20:59.065859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:20:59.073062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:59.073189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30407de-4cdf-46ef-9f07-abcfc4fdfaa9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:59.131664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:59.156584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:59.176191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58d702-cfc1-48e3-b529-63cb58601567 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:59.176645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30407de-4cdf-46ef-9f07-abcfc4fdfaa9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:59.210510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0b486-26e3-4c26-9ca2-f42b3e601a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:59.211021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3bc3d-fd1f-47c7-9841-530e6a36618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:59.211154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0b486-26e3-4c26-9ca2-f42b3e601a2c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:59.211430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0b486-26e3-4c26-9ca2-f42b3e601a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:59.211692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bc3d-fd1f-47c7-9841-530e6a36618d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:59.211775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0b486-26e3-4c26-9ca2-f42b3e601a2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:59.211973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bc3d-fd1f-47c7-9841-530e6a36618d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:59.212239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bc3d-fd1f-47c7-9841-530e6a36618d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:59.212427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0b486-26e3-4c26-9ca2-f42b3e601a2c None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.domain.id=default, user.id=13d95fead9ed4b918730e2198000e5e5) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:20:59.212738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0b486-26e3-4c26-9ca2-f42b3e601a2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:59.212886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3bc3d-fd1f-47c7-9841-530e6a36618d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:20:59.213184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3bc3d-fd1f-47c7-9841-530e6a36618d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:59.214189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0b486-26e3-4c26-9ca2-f42b3e601a2c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:59.214560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3bc3d-fd1f-47c7-9841-530e6a36618d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:59.221357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Aug 31 04:20:59 2022] GET /identity/v3/role_assignments?user.id=13d95fead9ed4b918730e2198000e5e5&scope.domain.id=default&role.id=241e99d365564c0fa95cb712f4004785 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:20:59.234309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:01.310154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ab384-b8ba-4c7d-b63b-f22dd434f951 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:01.310659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ab384-b8ba-4c7d-b63b-f22dd434f951 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:01.310756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ab384-b8ba-4c7d-b63b-f22dd434f951 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:01.312383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:01.324413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9aa34-721b-498e-b555-423f1e08b9d5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:01.325284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9aa34-721b-498e-b555-423f1e08b9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:01.325820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9aa34-721b-498e-b555-423f1e08b9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:01.349289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8df9aa34-721b-498e-b555-423f1e08b9d5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:01.350561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8df9aa34-721b-498e-b555-423f1e08b9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:01.440690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:01.535216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0dda-5e3d-4753-852b-c999482ef16e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:01.535946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0dda-5e3d-4753-852b-c999482ef16e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:01.536438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0dda-5e3d-4753-852b-c999482ef16e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:01.538447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:01.544096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4df8e3-13c1-4f15-b7b2-0622568eda9a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:01.544390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4df8e3-13c1-4f15-b7b2-0622568eda9a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:01.544672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4df8e3-13c1-4f15-b7b2-0622568eda9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:01.571687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5e4df8e3-13c1-4f15-b7b2-0622568eda9a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:01.572878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e4df8e3-13c1-4f15-b7b2-0622568eda9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:01.624955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:01.653619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 12/25] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:01.788905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e5581-e88a-4275-9ee4-3a0ee6e272e4 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:01.789576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e5581-e88a-4275-9ee4-3a0ee6e272e4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:01.790058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e5581-e88a-4275-9ee4-3a0ee6e272e4 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:01.791643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 14/26] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:01.797053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c11cc-7fb4-4899-97b0-a94b88c84864 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:01.797598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c11cc-7fb4-4899-97b0-a94b88c84864 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:01.798082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c11cc-7fb4-4899-97b0-a94b88c84864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:01.827778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7b9c11cc-7fb4-4899-97b0-a94b88c84864 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:01.829352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7b9c11cc-7fb4-4899-97b0-a94b88c84864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:01.878399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e5581-e88a-4275-9ee4-3a0ee6e272e4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:01.946742 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 13/27] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:02.044608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e5581-e88a-4275-9ee4-3a0ee6e272e4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:02.069096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4113e-29d5-4d91-b7ba-7adf77a40044 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:02.069406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4113e-29d5-4d91-b7ba-7adf77a40044 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:02.069717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4113e-29d5-4d91-b7ba-7adf77a40044 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:02.071126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 14/28] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:02.078857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d8d72-3e76-4945-81e9-50c78caf1e66 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:02.078857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d8d72-3e76-4945-81e9-50c78caf1e66 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:02.078857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d8d72-3e76-4945-81e9-50c78caf1e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:02.119984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0a9d8d72-3e76-4945-81e9-50c78caf1e66 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:02.121573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0a9d8d72-3e76-4945-81e9-50c78caf1e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:02.131419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a567e-894b-4e75-b620-2967cddc5284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:02.132406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a567e-894b-4e75-b620-2967cddc5284 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:02.132897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a567e-894b-4e75-b620-2967cddc5284 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:02.133347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a567e-894b-4e75-b620-2967cddc5284 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:02.161801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a567e-894b-4e75-b620-2967cddc5284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0b0a567e-894b-4e75-b620-2967cddc5284 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:02.167074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:02.167884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:02.168631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:02.174096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 15/29] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:02.175232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a567e-894b-4e75-b620-2967cddc5284 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:02.183057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:02.245576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 15/30] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:02.246214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d8d72-3e76-4945-81e9-50c78caf1e66 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:02.335825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:02.354843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a567e-894b-4e75-b620-2967cddc5284 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:02.390707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:02.391582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:02.392081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:02.392514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:02.404001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:02.405694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:21:02.422790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:02.423823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:21:02.424693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:21:02.424693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:21:02.424693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:02.426269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 16/31] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:02.431365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:02.471318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d8d72-3e76-4945-81e9-50c78caf1e66 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:02.492036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:02.539194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49674a0-8226-4335-b0c2-8a2f1c201962 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:02.550442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:02.551108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:02.551453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:02.551650 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:02.552369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:02.552676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:02.554392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:02.576180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 16/32] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:02.577437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:02.629567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f53d-94a9-48f2-9334-2f7f246959c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:02.630204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f53d-94a9-48f2-9334-2f7f246959c1 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:02.630491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f53d-94a9-48f2-9334-2f7f246959c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:02.630794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f53d-94a9-48f2-9334-2f7f246959c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:02.631592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f53d-94a9-48f2-9334-2f7f246959c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:02.633003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f53d-94a9-48f2-9334-2f7f246959c1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:02.713813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:02.717731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3760-9db4-4099-a71b-0f64d256fa82 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:02.754603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:02.755517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:02.755942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:02.756522 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:02.757618 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:02.758056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:02.761959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:02.774963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 17/33] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:02.776205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:02.782565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 17/34] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 31 04:21:02 2022] POST /identity/v3/roles => generated 225 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:02.786971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f53d-94a9-48f2-9334-2f7f246959c1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:02.947793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad489d85-9f5c-450b-b5c9-14b34a0fdcf1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:02.967619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f53d-94a9-48f2-9334-2f7f246959c1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:02.984199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bc12a-42fd-47b5-9a5e-679231617645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:02.984839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bc12a-42fd-47b5-9a5e-679231617645 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:02.985121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bc12a-42fd-47b5-9a5e-679231617645 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:02.985397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bc12a-42fd-47b5-9a5e-679231617645 None admin] PATH_INFO: `/v3/users/13d95fead9ed4b918730e2198000e5e5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:02.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bc12a-42fd-47b5-9a5e-679231617645 None admin] RBAC: Authorizing `identity:get_user(user_id=13d95fead9ed4b918730e2198000e5e5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:02.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bc12a-42fd-47b5-9a5e-679231617645 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:02.991320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 18/35] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:02 2022] GET /identity/v3/users/13d95fead9ed4b918730e2198000e5e5 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:02.992348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bc12a-42fd-47b5-9a5e-679231617645 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:02.999862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:03.028853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9649e-5742-4551-a7b1-9c9febaeb7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:03.029493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9649e-5742-4551-a7b1-9c9febaeb7d3 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.029778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9649e-5742-4551-a7b1-9c9febaeb7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.030051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9649e-5742-4551-a7b1-9c9febaeb7d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.030754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9649e-5742-4551-a7b1-9c9febaeb7d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:03.032047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:03.032299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9649e-5742-4551-a7b1-9c9febaeb7d3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:03.062912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 18/36] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 31 04:21:02 2022] POST /identity/v3/roles => generated 219 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:03.065208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e121ad-d6c7-4756-b4ba-91f00e55a95b None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.065504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e121ad-d6c7-4756-b4ba-91f00e55a95b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.065783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e121ad-d6c7-4756-b4ba-91f00e55a95b None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.071035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 19/37] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:03.071035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e121ad-d6c7-4756-b4ba-91f00e55a95b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:03.140784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bc12a-42fd-47b5-9a5e-679231617645 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:03.179990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f965257-a1c1-4261-a3b9-5f07b2ba1418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:03.180622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f965257-a1c1-4261-a3b9-5f07b2ba1418 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.180916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f965257-a1c1-4261-a3b9-5f07b2ba1418 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.181171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f965257-a1c1-4261-a3b9-5f07b2ba1418 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.181871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f965257-a1c1-4261-a3b9-5f07b2ba1418 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:03.183398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f965257-a1c1-4261-a3b9-5f07b2ba1418 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:03.188844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e121ad-d6c7-4756-b4ba-91f00e55a95b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:03.230701 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acf294-7c86-4490-8680-5e4f66cbe680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:03.231363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acf294-7c86-4490-8680-5e4f66cbe680 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.231643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acf294-7c86-4490-8680-5e4f66cbe680 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.231915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acf294-7c86-4490-8680-5e4f66cbe680 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.233131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acf294-7c86-4490-8680-5e4f66cbe680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:03.234582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acf294-7c86-4490-8680-5e4f66cbe680 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:03.238910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 20/38] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:03.238910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679e46-c35e-4658-95cf-1526c3a4270c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.238910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679e46-c35e-4658-95cf-1526c3a4270c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.238910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679e46-c35e-4658-95cf-1526c3a4270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.256728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b3679e46-c35e-4658-95cf-1526c3a4270c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:03.258213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b3679e46-c35e-4658-95cf-1526c3a4270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:03.298838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:03.324217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 19/39] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:21:02 2022] POST /identity/v3/projects => generated 301 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:03.334913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59296739-805c-4763-b5f5-6f434ce63160 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.334913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59296739-805c-4763-b5f5-6f434ce63160 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.334913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59296739-805c-4763-b5f5-6f434ce63160 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.335211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 21/40] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:03.335211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3679e46-c35e-4658-95cf-1526c3a4270c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:03.335366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 20/41] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:03.338409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff82160-aeff-411f-a4fc-69a94c12edae None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.339051 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff82160-aeff-411f-a4fc-69a94c12edae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.339051 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff82160-aeff-411f-a4fc-69a94c12edae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.362553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bff82160-aeff-411f-a4fc-69a94c12edae None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:03.364840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bff82160-aeff-411f-a4fc-69a94c12edae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:03.454755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 21/42] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:03.475507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3679e46-c35e-4658-95cf-1526c3a4270c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:03.514062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade68bc-3f55-453d-8b3c-bf8d8c86085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:03.514663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade68bc-3f55-453d-8b3c-bf8d8c86085e None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.515002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade68bc-3f55-453d-8b3c-bf8d8c86085e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.515281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade68bc-3f55-453d-8b3c-bf8d8c86085e None admin] PATH_INFO: `/v3/domains/default/users/13d95fead9ed4b918730e2198000e5e5/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.517212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade68bc-3f55-453d-8b3c-bf8d8c86085e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=13d95fead9ed4b918730e2198000e5e5, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:03.518674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade68bc-3f55-453d-8b3c-bf8d8c86085e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:03.539130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1361 bytes} [Wed Aug 31 04:21:03 2022] PUT /identity/v3/domains/default/users/13d95fead9ed4b918730e2198000e5e5/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:03.601303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff82160-aeff-411f-a4fc-69a94c12edae None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:03.612967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:03.714372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff82160-aeff-411f-a4fc-69a94c12edae None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:03.732554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade68bc-3f55-453d-8b3c-bf8d8c86085e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:03.751161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e3561-a8be-4e49-bb31-182806e8876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:03.751812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e3561-a8be-4e49-bb31-182806e8876e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.752094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e3561-a8be-4e49-bb31-182806e8876e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.752367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e3561-a8be-4e49-bb31-182806e8876e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.753473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e3561-a8be-4e49-bb31-182806e8876e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:03.755213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e3561-a8be-4e49-bb31-182806e8876e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:03.756998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 22/44] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:03.762620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e3561-a8be-4e49-bb31-182806e8876e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:03.769891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:03.841910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade68bc-3f55-453d-8b3c-bf8d8c86085e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:03.848575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:03.875350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e3561-a8be-4e49-bb31-182806e8876e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:03.915542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46361-bd74-44c2-b4d6-4a35ec4fb8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:03.916175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46361-bd74-44c2-b4d6-4a35ec4fb8aa None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.916456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46361-bd74-44c2-b4d6-4a35ec4fb8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.916729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46361-bd74-44c2-b4d6-4a35ec4fb8aa None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.917421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a46361-bd74-44c2-b4d6-4a35ec4fb8aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:03.918974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a46361-bd74-44c2-b4d6-4a35ec4fb8aa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:03.951467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 23/45] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:21:03 2022] POST /identity/v3/projects => generated 312 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:21:03.991778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c153e28-d3d8-4aa9-88ed-26e6e961221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:03.992411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153e28-d3d8-4aa9-88ed-26e6e961221a None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:03.992691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153e28-d3d8-4aa9-88ed-26e6e961221a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:03.992964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153e28-d3d8-4aa9-88ed-26e6e961221a None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:03.993659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c153e28-d3d8-4aa9-88ed-26e6e961221a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:03.995132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c153e28-d3d8-4aa9-88ed-26e6e961221a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:04.013512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 31 04:21:03 2022] POST /identity/v3/roles => generated 223 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:21:05.440845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421ace5-4126-4d70-96b6-674f370818e3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:05.441355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421ace5-4126-4d70-96b6-674f370818e3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:05.441355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421ace5-4126-4d70-96b6-674f370818e3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:05.442657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:05.447919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b567d0-292f-44a3-9d06-0a971c8e35c6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:05.448341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b567d0-292f-44a3-9d06-0a971c8e35c6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:05.448518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b567d0-292f-44a3-9d06-0a971c8e35c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:05.468364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-68b567d0-292f-44a3-9d06-0a971c8e35c6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:05.469570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-68b567d0-292f-44a3-9d06-0a971c8e35c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:05.540646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:05.818608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421ace5-4126-4d70-96b6-674f370818e3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:05.861304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b439-8e60-49ac-95ce-5a4746194887 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:05.861594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b439-8e60-49ac-95ce-5a4746194887 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:05.861884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b439-8e60-49ac-95ce-5a4746194887 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:05.863363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 25/49] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:05.868987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2d306-cd02-4b99-a2f7-95747cd6c732 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:05.869277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2d306-cd02-4b99-a2f7-95747cd6c732 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:05.869556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2d306-cd02-4b99-a2f7-95747cd6c732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:05.889361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bae2d306-cd02-4b99-a2f7-95747cd6c732 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:05.890572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bae2d306-cd02-4b99-a2f7-95747cd6c732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:05.909002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:05.926082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421ace5-4126-4d70-96b6-674f370818e3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:05.953541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:05.962605 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 26/50] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:05.962917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c07916-3ff6-4993-af65-e6a30ab0c9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:05.963713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c07916-3ff6-4993-af65-e6a30ab0c9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:05.964001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c07916-3ff6-4993-af65-e6a30ab0c9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:05.964273 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c07916-3ff6-4993-af65-e6a30ab0c9a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:05.977538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c07916-3ff6-4993-af65-e6a30ab0c9a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-58c07916-3ff6-4993-af65-e6a30ab0c9a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:05.983520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:05.984440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:05.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 25/51] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:05.991010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2d306-cd02-4b99-a2f7-95747cd6c732 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:06.094276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2d306-cd02-4b99-a2f7-95747cd6c732 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:06.127562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:06.128166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:06.128445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:06.128714 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:06.129375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:06.129676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:06.131106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:06.139619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 27/52] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:06.145128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c07916-3ff6-4993-af65-e6a30ab0c9a4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:06.222895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:06.242985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:06.247348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c07916-3ff6-4993-af65-e6a30ab0c9a4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:06.281468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553e13d-4b2f-48b2-bae3-7c4b56cbc9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:06.282103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553e13d-4b2f-48b2-bae3-7c4b56cbc9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:06.282381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553e13d-4b2f-48b2-bae3-7c4b56cbc9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:06.282651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553e13d-4b2f-48b2-bae3-7c4b56cbc9d3 None admin] PATH_INFO: `/v3/users/13d95fead9ed4b918730e2198000e5e5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:06.283849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553e13d-4b2f-48b2-bae3-7c4b56cbc9d3 None admin] RBAC: Authorizing `identity:get_user(user_id=13d95fead9ed4b918730e2198000e5e5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:06.285319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553e13d-4b2f-48b2-bae3-7c4b56cbc9d3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:06.291198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 26/53] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:06 2022] GET /identity/v3/users/13d95fead9ed4b918730e2198000e5e5 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:06.296847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553e13d-4b2f-48b2-bae3-7c4b56cbc9d3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:06.325968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d8d6b-dc3f-4637-9dd1-44a94e7d3236 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:06.354995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:06.361493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806e7e3-4970-40a9-be55-edd2542f8773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:06.362137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806e7e3-4970-40a9-be55-edd2542f8773 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:06.362421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806e7e3-4970-40a9-be55-edd2542f8773 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:06.362694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806e7e3-4970-40a9-be55-edd2542f8773 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:06.363850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806e7e3-4970-40a9-be55-edd2542f8773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:06.365333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806e7e3-4970-40a9-be55-edd2542f8773 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:06.367043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 28/54] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:06.372589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806e7e3-4970-40a9-be55-edd2542f8773 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:06.401938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553e13d-4b2f-48b2-bae3-7c4b56cbc9d3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:06.435054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457e9c-c086-46f9-ac0b-f02c7fb66e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:06.435687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457e9c-c086-46f9-ac0b-f02c7fb66e4f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:06.435969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457e9c-c086-46f9-ac0b-f02c7fb66e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:06.436242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457e9c-c086-46f9-ac0b-f02c7fb66e4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:06.437379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0457e9c-c086-46f9-ac0b-f02c7fb66e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:06.438888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0457e9c-c086-46f9-ac0b-f02c7fb66e4f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:06.440867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 27/55] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:06.446706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457e9c-c086-46f9-ac0b-f02c7fb66e4f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:06.455962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:06.474587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806e7e3-4970-40a9-be55-edd2542f8773 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:06.516415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b415245-3d3d-4417-b0b0-688c95873287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:06.517051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b415245-3d3d-4417-b0b0-688c95873287 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:06.517339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b415245-3d3d-4417-b0b0-688c95873287 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:06.517614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b415245-3d3d-4417-b0b0-688c95873287 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:06.518282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b415245-3d3d-4417-b0b0-688c95873287 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:06.519837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b415245-3d3d-4417-b0b0-688c95873287 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:06.553041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 29/56] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Aug 31 04:21:06 2022] POST /identity/v3/projects => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:21:06.669391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457e9c-c086-46f9-ac0b-f02c7fb66e4f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:06.695852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:06.708172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89f213-9706-49d1-9b6f-6748683b6c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:06.708797 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89f213-9706-49d1-9b6f-6748683b6c7d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:06.709074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89f213-9706-49d1-9b6f-6748683b6c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:06.709344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89f213-9706-49d1-9b6f-6748683b6c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:06.710295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89f213-9706-49d1-9b6f-6748683b6c7d None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.domain.id=default, user.id=13d95fead9ed4b918730e2198000e5e5) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:06.710597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89f213-9706-49d1-9b6f-6748683b6c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:06.712090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89f213-9706-49d1-9b6f-6748683b6c7d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:06.720889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 28/57] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Aug 31 04:21:06 2022] GET /identity/v3/role_assignments?user.id=13d95fead9ed4b918730e2198000e5e5&scope.domain.id=default&role.id=241e99d365564c0fa95cb712f4004785 => generated 530 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:08.426595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9156fe-7484-47a2-b269-8e71a0688025 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:08.427141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9156fe-7484-47a2-b269-8e71a0688025 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:08.427141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9156fe-7484-47a2-b269-8e71a0688025 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:08.428472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 30/58] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:08.434769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38006cd3-5e53-41b4-877b-84c61b375095 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:08.435449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38006cd3-5e53-41b4-877b-84c61b375095 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:08.435675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38006cd3-5e53-41b4-877b-84c61b375095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:08.463565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-38006cd3-5e53-41b4-877b-84c61b375095 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:08.464782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38006cd3-5e53-41b4-877b-84c61b375095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:08.534591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 29/59] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:08.810115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9156fe-7484-47a2-b269-8e71a0688025 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:08.929374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9156fe-7484-47a2-b269-8e71a0688025 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:08.964554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325d391-77b9-4152-b6c3-7bf19bb6c95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:08.965185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325d391-77b9-4152-b6c3-7bf19bb6c95d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:08.965460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325d391-77b9-4152-b6c3-7bf19bb6c95d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:08.965739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325d391-77b9-4152-b6c3-7bf19bb6c95d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:08.967012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325d391-77b9-4152-b6c3-7bf19bb6c95d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:08.968492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325d391-77b9-4152-b6c3-7bf19bb6c95d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:08.970059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 31/60] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:08.976691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38006cd3-5e53-41b4-877b-84c61b375095 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:09.093802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38006cd3-5e53-41b4-877b-84c61b375095 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:09.111062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:09.128765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf11a0-5cc1-41a5-a759-bd7178c492c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:09.129380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf11a0-5cc1-41a5-a759-bd7178c492c3 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:09.129655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf11a0-5cc1-41a5-a759-bd7178c492c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:09.129935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf11a0-5cc1-41a5-a759-bd7178c492c3 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:09.130614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf11a0-5cc1-41a5-a759-bd7178c492c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:09.132116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf11a0-5cc1-41a5-a759-bd7178c492c3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:09.162392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 30/61] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:21:08 2022] POST /identity/v3/users => generated 275 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:12.226101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6271f80-f08f-489c-a6d9-1042f5fdc3b5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.226610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6271f80-f08f-489c-a6d9-1042f5fdc3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.226682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6271f80-f08f-489c-a6d9-1042f5fdc3b5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.228517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 32/62] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:12.233647 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b364-6bc5-4caf-9401-33c0c8485f37 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.234055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b364-6bc5-4caf-9401-33c0c8485f37 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.234201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b364-6bc5-4caf-9401-33c0c8485f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.263823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9321b364-6bc5-4caf-9401-33c0c8485f37 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:12.265322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9321b364-6bc5-4caf-9401-33c0c8485f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:12.308080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5eb95-e70d-4674-9a49-600a38b0e121 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.308710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5eb95-e70d-4674-9a49-600a38b0e121 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.309265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5eb95-e70d-4674-9a49-600a38b0e121 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.310834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 33/63] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:12.316381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311db959-d8d9-409b-bf51-d071b9c7e45a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.316908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311db959-d8d9-409b-bf51-d071b9c7e45a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.317378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311db959-d8d9-409b-bf51-d071b9c7e45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.329237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:12.347111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-311db959-d8d9-409b-bf51-d071b9c7e45a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:12.348633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-311db959-d8d9-409b-bf51-d071b9c7e45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:12.370264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 31/64] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:12.432425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc05c9-a620-4e39-bd53-90c347f80aca None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.434995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc05c9-a620-4e39-bd53-90c347f80aca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.435465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc05c9-a620-4e39-bd53-90c347f80aca None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.437007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 32/65] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:12.448145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782d81-2edd-42d2-9a2c-1b8da82257c6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.448736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782d81-2edd-42d2-9a2c-1b8da82257c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.449197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782d81-2edd-42d2-9a2c-1b8da82257c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.482839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bc782d81-2edd-42d2-9a2c-1b8da82257c6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:12.482839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc782d81-2edd-42d2-9a2c-1b8da82257c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:12.505675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 34/66] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:12.582145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 33/67] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:12.741018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311db959-d8d9-409b-bf51-d071b9c7e45a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:12.784250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782d81-2edd-42d2-9a2c-1b8da82257c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:12.830903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:12.846929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311db959-d8d9-409b-bf51-d071b9c7e45a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:12.881813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5dd09-597d-467d-8378-e5a108bfb8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:12.882455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5dd09-597d-467d-8378-e5a108bfb8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.882758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5dd09-597d-467d-8378-e5a108bfb8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.883024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5dd09-597d-467d-8378-e5a108bfb8a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.885220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:12.894351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5dd09-597d-467d-8378-e5a108bfb8a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-47d5dd09-597d-467d-8378-e5a108bfb8a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:12.899371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:12.900190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:12.900914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 35/68] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:12.902659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71cd8-3966-482c-b6d2-6b73915eaa25 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.902964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71cd8-3966-482c-b6d2-6b73915eaa25 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.903235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71cd8-3966-482c-b6d2-6b73915eaa25 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.904644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 36/69] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:12.907905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71cd8-3966-482c-b6d2-6b73915eaa25 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:12.925869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782d81-2edd-42d2-9a2c-1b8da82257c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:12.960158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa24c3-3361-4325-bcaa-80cbed6b05f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:12.960791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa24c3-3361-4325-bcaa-80cbed6b05f9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:12.961071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa24c3-3361-4325-bcaa-80cbed6b05f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:12.961369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa24c3-3361-4325-bcaa-80cbed6b05f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:12.971536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa24c3-3361-4325-bcaa-80cbed6b05f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-53fa24c3-3361-4325-bcaa-80cbed6b05f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:12.976450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:12.977369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:12.978098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 34/70] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:12.978743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa24c3-3361-4325-bcaa-80cbed6b05f9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.012252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71cd8-3966-482c-b6d2-6b73915eaa25 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.046969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6173d-f6b6-4eba-a2a4-112dc1251ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.047576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6173d-f6b6-4eba-a2a4-112dc1251ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.047880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6173d-f6b6-4eba-a2a4-112dc1251ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.048152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6173d-f6b6-4eba-a2a4-112dc1251ab4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.061764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6173d-f6b6-4eba-a2a4-112dc1251ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b7b6173d-f6b6-4eba-a2a4-112dc1251ab4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:13.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:13.065719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:13.066580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 37/71] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:13.067246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4410a36-0bf2-4da9-96c3-77e3d50a1ba9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.067820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4410a36-0bf2-4da9-96c3-77e3d50a1ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.067820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4410a36-0bf2-4da9-96c3-77e3d50a1ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.075803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:13.082952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa24c3-3361-4325-bcaa-80cbed6b05f9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.086786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e4410a36-0bf2-4da9-96c3-77e3d50a1ba9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:13.088065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e4410a36-0bf2-4da9-96c3-77e3d50a1ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:13.116491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158af41c-f7b5-4a1e-bc9d-899d7853bf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.117043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158af41c-f7b5-4a1e-bc9d-899d7853bf7b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.117326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158af41c-f7b5-4a1e-bc9d-899d7853bf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.117621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158af41c-f7b5-4a1e-bc9d-899d7853bf7b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.118293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158af41c-f7b5-4a1e-bc9d-899d7853bf7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:13.118594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158af41c-f7b5-4a1e-bc9d-899d7853bf7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:13.120038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158af41c-f7b5-4a1e-bc9d-899d7853bf7b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:13.128662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 35/72] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:13.129963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a9f9-7daa-4f25-bfed-b3e1a9b45951 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.130249 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a9f9-7daa-4f25-bfed-b3e1a9b45951 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.130531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a9f9-7daa-4f25-bfed-b3e1a9b45951 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.131750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 36/73] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:13.135541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a9f9-7daa-4f25-bfed-b3e1a9b45951 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.139240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:13.158312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 38/74] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:13.159183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4410a36-0bf2-4da9-96c3-77e3d50a1ba9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.247828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a9f9-7daa-4f25-bfed-b3e1a9b45951 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.263657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4410a36-0bf2-4da9-96c3-77e3d50a1ba9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.284888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.285517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.285799 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.286072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.286766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:13.287073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:13.288468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:13.297837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 37/75] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:13.298634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.303284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e788b0-6f6f-4d5a-b648-fcc6ec5d0f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.303911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e788b0-6f6f-4d5a-b648-fcc6ec5d0f77 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.304193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e788b0-6f6f-4d5a-b648-fcc6ec5d0f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.304469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e788b0-6f6f-4d5a-b648-fcc6ec5d0f77 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.305181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e788b0-6f6f-4d5a-b648-fcc6ec5d0f77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:13.305492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e788b0-6f6f-4d5a-b648-fcc6ec5d0f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:13.306952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e788b0-6f6f-4d5a-b648-fcc6ec5d0f77 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:13.316060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 39/76] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:13.317058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9962d14-2c10-4b55-b84c-25bff41fd28c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.317344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9962d14-2c10-4b55-b84c-25bff41fd28c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.317619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9962d14-2c10-4b55-b84c-25bff41fd28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.328342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:13.336134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f9962d14-2c10-4b55-b84c-25bff41fd28c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:13.337429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f9962d14-2c10-4b55-b84c-25bff41fd28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:13.382808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:13.408629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 40/77] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:13.409557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9962d14-2c10-4b55-b84c-25bff41fd28c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.414803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df8d18-7073-4474-b064-e63d8f2cda04 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.455456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216fe72-7c5c-4c36-91da-91e18ee52d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.456159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216fe72-7c5c-4c36-91da-91e18ee52d01 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.456446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216fe72-7c5c-4c36-91da-91e18ee52d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.456719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216fe72-7c5c-4c36-91da-91e18ee52d01 None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.473073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216fe72-7c5c-4c36-91da-91e18ee52d01 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:13.474550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216fe72-7c5c-4c36-91da-91e18ee52d01 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:13.476225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 38/78] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:13.479276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216fe72-7c5c-4c36-91da-91e18ee52d01 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.517026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9962d14-2c10-4b55-b84c-25bff41fd28c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.558148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:13.560171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9b445-5a70-4f0b-8229-31448fac8fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.561389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9b445-5a70-4f0b-8229-31448fac8fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.561910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9b445-5a70-4f0b-8229-31448fac8fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.562441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9b445-5a70-4f0b-8229-31448fac8fd5 None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.564432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9b445-5a70-4f0b-8229-31448fac8fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:13.566561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9b445-5a70-4f0b-8229-31448fac8fd5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:13.568605 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 41/79] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:13.573998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9b445-5a70-4f0b-8229-31448fac8fd5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.582840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216fe72-7c5c-4c36-91da-91e18ee52d01 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.617770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfe1c1-f107-40b6-ab3d-8a78deeb0e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.618477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfe1c1-f107-40b6-ab3d-8a78deeb0e89 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.618780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfe1c1-f107-40b6-ab3d-8a78deeb0e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.619059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfe1c1-f107-40b6-ab3d-8a78deeb0e89 None admin] PATH_INFO: `/v3/users/13d95fead9ed4b918730e2198000e5e5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.620203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfe1c1-f107-40b6-ab3d-8a78deeb0e89 None admin] RBAC: Authorizing `identity:get_user(user_id=13d95fead9ed4b918730e2198000e5e5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:13.621198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:13.621696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfe1c1-f107-40b6-ab3d-8a78deeb0e89 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:13.623367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 39/80] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/users/13d95fead9ed4b918730e2198000e5e5 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:13.624259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfe1c1-f107-40b6-ab3d-8a78deeb0e89 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.679578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9b445-5a70-4f0b-8229-31448fac8fd5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.715367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2a254-9c19-4f29-b442-b7fd02d2161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.715994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2a254-9c19-4f29-b442-b7fd02d2161d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.716272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2a254-9c19-4f29-b442-b7fd02d2161d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.716542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2a254-9c19-4f29-b442-b7fd02d2161d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.726449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2a254-9c19-4f29-b442-b7fd02d2161d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:13.727319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfe1c1-f107-40b6-ab3d-8a78deeb0e89 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2ba2a254-9c19-4f29-b442-b7fd02d2161d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:13.731534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:13.732371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:13.733169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 42/81] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:13.733854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2a254-9c19-4f29-b442-b7fd02d2161d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.762797 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde156c1-b5de-40b8-ac72-c4fd48d0556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.763417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde156c1-b5de-40b8-ac72-c4fd48d0556a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.763698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde156c1-b5de-40b8-ac72-c4fd48d0556a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.764022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde156c1-b5de-40b8-ac72-c4fd48d0556a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.778476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:13.830872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde156c1-b5de-40b8-ac72-c4fd48d0556a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:13.831694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2a254-9c19-4f29-b442-b7fd02d2161d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:13.832364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde156c1-b5de-40b8-ac72-c4fd48d0556a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:13.833972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:13.867142 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2abd0-1045-4fa1-beff-bdf435707480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:13.867821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2abd0-1045-4fa1-beff-bdf435707480 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:13.868099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2abd0-1045-4fa1-beff-bdf435707480 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:13.868369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2abd0-1045-4fa1-beff-bdf435707480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:13.931065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 40/82] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:13.932095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde156c1-b5de-40b8-ac72-c4fd48d0556a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:13.932905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2abd0-1045-4fa1-beff-bdf435707480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:13.934398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2abd0-1045-4fa1-beff-bdf435707480 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:13.979001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.033379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 43/83] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:14.033993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde156c1-b5de-40b8-ac72-c4fd48d0556a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.035620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2abd0-1045-4fa1-beff-bdf435707480 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.038766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.073856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3676e3-f0f7-469a-8190-0277f14a1fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.074477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3676e3-f0f7-469a-8190-0277f14a1fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.074789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3676e3-f0f7-469a-8190-0277f14a1fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.075063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3676e3-f0f7-469a-8190-0277f14a1fa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.136084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2abd0-1045-4fa1-beff-bdf435707480 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.142477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3676e3-f0f7-469a-8190-0277f14a1fa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.143993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3676e3-f0f7-469a-8190-0277f14a1fa5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.169241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434c707-898a-49b9-b039-522855264b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.169881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434c707-898a-49b9-b039-522855264b78 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.170166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434c707-898a-49b9-b039-522855264b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.170438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434c707-898a-49b9-b039-522855264b78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.171612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434c707-898a-49b9-b039-522855264b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.173082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434c707-898a-49b9-b039-522855264b78 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.174884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 44/84] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:14.175454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434c707-898a-49b9-b039-522855264b78 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.192354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.242304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.247346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 41/85] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:14.250146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3676e3-f0f7-469a-8190-0277f14a1fa5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.272165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434c707-898a-49b9-b039-522855264b78 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.306540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.307188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.307470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.307742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.308412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:14.308717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.310131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.318400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 45/86] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:14.323525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.352425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3676e3-f0f7-469a-8190-0277f14a1fa5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.386225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e3c83-82d8-413a-8f13-4b0ea748e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.386895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e3c83-82d8-413a-8f13-4b0ea748e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.387163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e3c83-82d8-413a-8f13-4b0ea748e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.387437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e3c83-82d8-413a-8f13-4b0ea748e0c2 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.388567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e3c83-82d8-413a-8f13-4b0ea748e0c2 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.390033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e3c83-82d8-413a-8f13-4b0ea748e0c2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.391719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 42/87] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:14.392458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e3c83-82d8-413a-8f13-4b0ea748e0c2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.408523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.418493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b74f-0742-4b26-a0bf-2eee477d7a6d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.453250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a149a-8655-43e4-99c0-f4607de188a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.453804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a149a-8655-43e4-99c0-f4607de188a7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.454091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a149a-8655-43e4-99c0-f4607de188a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.454407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a149a-8655-43e4-99c0-f4607de188a7 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.455548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a149a-8655-43e4-99c0-f4607de188a7 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.457008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a149a-8655-43e4-99c0-f4607de188a7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.458656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 46/88] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:14.462895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a149a-8655-43e4-99c0-f4607de188a7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.462895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.497322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e3c83-82d8-413a-8f13-4b0ea748e0c2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.531387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42311bcd-9dfc-46bc-86a7-10d97c550810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.531999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42311bcd-9dfc-46bc-86a7-10d97c550810 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.532283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42311bcd-9dfc-46bc-86a7-10d97c550810 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.532560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42311bcd-9dfc-46bc-86a7-10d97c550810 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.533211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42311bcd-9dfc-46bc-86a7-10d97c550810 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.534649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42311bcd-9dfc-46bc-86a7-10d97c550810 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.556023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 43/89] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:21:14 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:14.556855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42311bcd-9dfc-46bc-86a7-10d97c550810 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.571685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a149a-8655-43e4-99c0-f4607de188a7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.617998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d20d4-ea26-48b6-a764-2720a3307934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.618809 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d20d4-ea26-48b6-a764-2720a3307934 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.619093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d20d4-ea26-48b6-a764-2720a3307934 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.619365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d20d4-ea26-48b6-a764-2720a3307934 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.620681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d20d4-ea26-48b6-a764-2720a3307934 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.622300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d20d4-ea26-48b6-a764-2720a3307934 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.624223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 47/90] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:14.630039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d20d4-ea26-48b6-a764-2720a3307934 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.641361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.681185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42311bcd-9dfc-46bc-86a7-10d97c550810 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.706098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.724049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857b2cf-cb0b-483a-810c-d28eada9497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.724714 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857b2cf-cb0b-483a-810c-d28eada9497e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.725028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857b2cf-cb0b-483a-810c-d28eada9497e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.725279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857b2cf-cb0b-483a-810c-d28eada9497e None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.726473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857b2cf-cb0b-483a-810c-d28eada9497e None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.728267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857b2cf-cb0b-483a-810c-d28eada9497e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.730450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 44/91] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:14.730663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857b2cf-cb0b-483a-810c-d28eada9497e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.752646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d20d4-ea26-48b6-a764-2720a3307934 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.791993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.792619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.792949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.793230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.793988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] RBAC: Adding query filter params (role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf, user.id=0d9b7166b08d4c81a876de0dd637906f) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:14.794356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.795859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.803212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 48/92] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/role_assignments?user.id=0d9b7166b08d4c81a876de0dd637906f&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=5bbcda16701c451f9419aefc4671840e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:14.804050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.852207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857b2cf-cb0b-483a-810c-d28eada9497e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.904366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.905059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.905343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.905616 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.906867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] RBAC: Adding query filter params (role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf, user.id=0d9b7166b08d4c81a876de0dd637906f) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:14.906867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.908293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:14.912266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:14.917024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 45/93] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/role_assignments?user.id=0d9b7166b08d4c81a876de0dd637906f&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=d06115dbab78488182bcb432bf9bec11 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:14.917426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:14.953714 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa13dd8-9214-44b0-9d7e-ca686e337b5a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:14.994971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:14.995836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:14.996262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:14.996695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:14.997590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf, user.id=13d95fead9ed4b918730e2198000e5e5) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:14.998043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:14.999808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:15.002092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:15.007131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 49/94] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/role_assignments?user.id=13d95fead9ed4b918730e2198000e5e5&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=241e99d365564c0fa95cb712f4004785 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:15.052548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd36bb1-5a6b-4866-bcb9-cc60b9d08628 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:15.104240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8832b-9f64-4a48-aab3-8883beb5951d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:15.104879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8832b-9f64-4a48-aab3-8883beb5951d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:15.105159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8832b-9f64-4a48-aab3-8883beb5951d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:15.105428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8832b-9f64-4a48-aab3-8883beb5951d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:15.192499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8832b-9f64-4a48-aab3-8883beb5951d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:15.194078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8832b-9f64-4a48-aab3-8883beb5951d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:15.199090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:15.300794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 46/95] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:15.306451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:15.438601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35805d-9442-4d67-b9e3-0a9ac3f78a4f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:15.488756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16aa52b-35e8-4aab-bc75-7a01e82cf070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:15.489410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa52b-35e8-4aab-bc75-7a01e82cf070 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:15.489688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa52b-35e8-4aab-bc75-7a01e82cf070 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:15.489962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa52b-35e8-4aab-bc75-7a01e82cf070 None admin] PATH_INFO: `/v3/projects/57244b4e47524ab28ec4382ca7fb13a9` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:15.491127 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16aa52b-35e8-4aab-bc75-7a01e82cf070 None admin] RBAC: Authorizing `identity:get_project(project_id=57244b4e47524ab28ec4382ca7fb13a9)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:15.492590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16aa52b-35e8-4aab-bc75-7a01e82cf070 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:15.494663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 50/96] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:15 2022] GET /identity/v3/projects/57244b4e47524ab28ec4382ca7fb13a9 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:15.500211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8832b-9f64-4a48-aab3-8883beb5951d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:15.615141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8832b-9f64-4a48-aab3-8883beb5951d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:15.647499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:15.668854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33abc2-2d06-40fe-bc69-1aeec283aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:15.669474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33abc2-2d06-40fe-bc69-1aeec283aa7c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:15.669753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33abc2-2d06-40fe-bc69-1aeec283aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:15.670026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33abc2-2d06-40fe-bc69-1aeec283aa7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:15.670789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33abc2-2d06-40fe-bc69-1aeec283aa7c None admin] RBAC: Adding query filter params (role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=57244b4e47524ab28ec4382ca7fb13a9, user.id=0d9b7166b08d4c81a876de0dd637906f) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:15.671098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33abc2-2d06-40fe-bc69-1aeec283aa7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:15.672611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33abc2-2d06-40fe-bc69-1aeec283aa7c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:15.678443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 47/97] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:15 2022] GET /identity/v3/role_assignments?user.id=0d9b7166b08d4c81a876de0dd637906f&scope.project.id=57244b4e47524ab28ec4382ca7fb13a9&role.id=5bbcda16701c451f9419aefc4671840e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:17.171208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48388b-8adf-4b77-8280-1d820118f6ad None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:17.172203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48388b-8adf-4b77-8280-1d820118f6ad None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:17.172688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48388b-8adf-4b77-8280-1d820118f6ad None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:17.174279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 51/98] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:17.181815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecaa53-2473-4d4a-844a-78f548eaaa2d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:17.182695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecaa53-2473-4d4a-844a-78f548eaaa2d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:17.183214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecaa53-2473-4d4a-844a-78f548eaaa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:17.191073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c1a4e-64b1-4ce0-9e12-2dcd896e88d1 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:17.191639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c1a4e-64b1-4ce0-9e12-2dcd896e88d1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:17.192130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c1a4e-64b1-4ce0-9e12-2dcd896e88d1 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:17.193793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 52/99] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:17.202097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692d779-293f-435d-854d-487e90526335 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:17.203010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692d779-293f-435d-854d-487e90526335 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:17.203496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692d779-293f-435d-854d-487e90526335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:17.224607 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a8ecaa53-2473-4d4a-844a-78f548eaaa2d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:17.230267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a8ecaa53-2473-4d4a-844a-78f548eaaa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:17.243789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d692d779-293f-435d-854d-487e90526335 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:17.245649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d692d779-293f-435d-854d-487e90526335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:17.347218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:17.354129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 53/100] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:17.381353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 48/101] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:17.845421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771ae02-17f3-400b-a94e-0f440cfe1614 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:17.845800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771ae02-17f3-400b-a94e-0f440cfe1614 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:17.845983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771ae02-17f3-400b-a94e-0f440cfe1614 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:17.847583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 54/102] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:17.860506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150e384-6f32-4608-95c9-e7459c69d6f7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:17.860862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150e384-6f32-4608-95c9-e7459c69d6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:17.861079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150e384-6f32-4608-95c9-e7459c69d6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:17.884783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9150e384-6f32-4608-95c9-e7459c69d6f7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:17.886002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9150e384-6f32-4608-95c9-e7459c69d6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:17.915704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b8be-de7d-4522-b9ee-e4922225f9e2 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:17.916022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b8be-de7d-4522-b9ee-e4922225f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:17.916262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b8be-de7d-4522-b9ee-e4922225f9e2 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:17.917622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 55/103] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:17.922693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebd069-9a97-41db-85ae-5c938847421c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:17.923020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebd069-9a97-41db-85ae-5c938847421c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:17.923299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebd069-9a97-41db-85ae-5c938847421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:17.956753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f3ebd069-9a97-41db-85ae-5c938847421c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:17.957975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f3ebd069-9a97-41db-85ae-5c938847421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:17.979773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 49/104] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:17.980656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150e384-6f32-4608-95c9-e7459c69d6f7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.010067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:18.106444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150e384-6f32-4608-95c9-e7459c69d6f7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:18.148502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3dafb-1b7d-4381-b53e-82a826128d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:18.149574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3dafb-1b7d-4381-b53e-82a826128d2d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.150199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3dafb-1b7d-4381-b53e-82a826128d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.150619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3dafb-1b7d-4381-b53e-82a826128d2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.163089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 56/105] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:18.163969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebd069-9a97-41db-85ae-5c938847421c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.164282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3dafb-1b7d-4381-b53e-82a826128d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-44b3dafb-1b7d-4381-b53e-82a826128d2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:18.170112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:18.171151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:18.171951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:18.173763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 50/106] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:18.176694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3103dc1-2f27-452e-aaeb-88c23acead1e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.177187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3103dc1-2f27-452e-aaeb-88c23acead1e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.177645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3103dc1-2f27-452e-aaeb-88c23acead1e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.179115 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 51/107] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:18.179965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3103dc1-2f27-452e-aaeb-88c23acead1e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.281245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:18.306464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3103dc1-2f27-452e-aaeb-88c23acead1e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:18.347813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a1670-4114-4554-9bed-1cefe4c52c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:18.348854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a1670-4114-4554-9bed-1cefe4c52c93 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.349281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a1670-4114-4554-9bed-1cefe4c52c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.350197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a1670-4114-4554-9bed-1cefe4c52c93 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.351143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a1670-4114-4554-9bed-1cefe4c52c93 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:18.351599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a1670-4114-4554-9bed-1cefe4c52c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:18.353618 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a1670-4114-4554-9bed-1cefe4c52c93 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:18.364851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 52/108] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:18.366293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e7a5-4b6e-47d2-bdf4-57f1f8c3b3bd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.366757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e7a5-4b6e-47d2-bdf4-57f1f8c3b3bd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.367172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e7a5-4b6e-47d2-bdf4-57f1f8c3b3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.390606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-24e5e7a5-4b6e-47d2-bdf4-57f1f8c3b3bd None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:18.392462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24e5e7a5-4b6e-47d2-bdf4-57f1f8c3b3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:18.399400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebd069-9a97-41db-85ae-5c938847421c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:18.435117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:18.439627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487b797-1e45-4420-9cf5-66c2fe3d9b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:18.440720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b797-1e45-4420-9cf5-66c2fe3d9b3e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.441212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b797-1e45-4420-9cf5-66c2fe3d9b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.441492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487b797-1e45-4420-9cf5-66c2fe3d9b3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.443313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487b797-1e45-4420-9cf5-66c2fe3d9b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:18.444494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487b797-1e45-4420-9cf5-66c2fe3d9b3e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:18.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 57/109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:18.447173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487b797-1e45-4420-9cf5-66c2fe3d9b3e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.484725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 53/110] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:18.485894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5e7a5-4b6e-47d2-bdf4-57f1f8c3b3bd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.574545 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487b797-1e45-4420-9cf5-66c2fe3d9b3e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:18.580370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:18.610953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5e7a5-4b6e-47d2-bdf4-57f1f8c3b3bd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:18.625261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb597d-5bc0-4a89-8be7-59999a0b80c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:18.625814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb597d-5bc0-4a89-8be7-59999a0b80c0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.626095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb597d-5bc0-4a89-8be7-59999a0b80c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.626368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb597d-5bc0-4a89-8be7-59999a0b80c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.637763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb597d-5bc0-4a89-8be7-59999a0b80c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-70fb597d-5bc0-4a89-8be7-59999a0b80c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:18.643219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:18.644623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 58/111] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:18.645823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb597d-5bc0-4a89-8be7-59999a0b80c0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.665689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fa5c-aaca-4867-96a2-f40dcc0cbdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:18.667184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fa5c-aaca-4867-96a2-f40dcc0cbdff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.667882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fa5c-aaca-4867-96a2-f40dcc0cbdff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.668468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fa5c-aaca-4867-96a2-f40dcc0cbdff None admin] PATH_INFO: `/v3/users/63ea915cb96141c684220c4df0efa829` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.686479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7fa5c-aaca-4867-96a2-f40dcc0cbdff None admin] RBAC: Authorizing `identity:get_user(user_id=63ea915cb96141c684220c4df0efa829)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:18.689767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7fa5c-aaca-4867-96a2-f40dcc0cbdff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:18.692121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 54/112] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:18 2022] GET /identity/v3/users/63ea915cb96141c684220c4df0efa829 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:18.693163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fa5c-aaca-4867-96a2-f40dcc0cbdff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.718819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:18.769334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb597d-5bc0-4a89-8be7-59999a0b80c0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:18.805516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fab0c-942d-4d9f-9352-e58ca610db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:18.806248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fab0c-942d-4d9f-9352-e58ca610db07 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.806530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fab0c-942d-4d9f-9352-e58ca610db07 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.806843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fab0c-942d-4d9f-9352-e58ca610db07 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.807501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fab0c-942d-4d9f-9352-e58ca610db07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:18.808970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fab0c-942d-4d9f-9352-e58ca610db07 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:18.813950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fa5c-aaca-4867-96a2-f40dcc0cbdff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:18.836963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 59/113] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:21:18 2022] POST /identity/v3/projects => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:18.837816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fab0c-942d-4d9f-9352-e58ca610db07 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.867635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e946d-474d-426d-bb9d-54ca733a0a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:18.868630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e946d-474d-426d-bb9d-54ca733a0a34 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:18.869124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e946d-474d-426d-bb9d-54ca733a0a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:18.869467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e946d-474d-426d-bb9d-54ca733a0a34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:18.871897 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:18.885552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e946d-474d-426d-bb9d-54ca733a0a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ad9e946d-474d-426d-bb9d-54ca733a0a34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:18.890968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:18.891895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:18.892630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 55/114] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:18.893247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e946d-474d-426d-bb9d-54ca733a0a34 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:18.957482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fab0c-942d-4d9f-9352-e58ca610db07 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:18.967141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:19.003986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e946d-474d-426d-bb9d-54ca733a0a34 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.004180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c722f30-4200-4850-a924-153cc317c288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.004836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c722f30-4200-4850-a924-153cc317c288 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.005099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c722f30-4200-4850-a924-153cc317c288 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.005380 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c722f30-4200-4850-a924-153cc317c288 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.006069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c722f30-4200-4850-a924-153cc317c288 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:19.006377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c722f30-4200-4850-a924-153cc317c288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.007816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c722f30-4200-4850-a924-153cc317c288 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.016398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 60/115] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.017753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a300b-a161-4fb6-a520-639a9cd119f0 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.017924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a300b-a161-4fb6-a520-639a9cd119f0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.018285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a300b-a161-4fb6-a520-639a9cd119f0 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.019671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 61/116] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:19.020237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a300b-a161-4fb6-a520-639a9cd119f0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.054448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df72f7-e898-402f-8498-578e777a18b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.055103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df72f7-e898-402f-8498-578e777a18b6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.055456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df72f7-e898-402f-8498-578e777a18b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.056067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df72f7-e898-402f-8498-578e777a18b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.127982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:19.131724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a300b-a161-4fb6-a520-639a9cd119f0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.139015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df72f7-e898-402f-8498-578e777a18b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.140478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df72f7-e898-402f-8498-578e777a18b6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.171275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaa61d-bde7-44b2-9f06-f69df7dd7442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.171918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaa61d-bde7-44b2-9f06-f69df7dd7442 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.172200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaa61d-bde7-44b2-9f06-f69df7dd7442 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.172479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaa61d-bde7-44b2-9f06-f69df7dd7442 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.183848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aaa61d-bde7-44b2-9f06-f69df7dd7442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-31aaa61d-bde7-44b2-9f06-f69df7dd7442 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:19.188718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:19.189548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 62/117] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:19.191180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaa61d-bde7-44b2-9f06-f69df7dd7442 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.222078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:19.248215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 56/118] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:19.253693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df72f7-e898-402f-8498-578e777a18b6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.299819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaa61d-bde7-44b2-9f06-f69df7dd7442 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.339096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296436e-db1a-4624-b23f-146d034903c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.339715 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296436e-db1a-4624-b23f-146d034903c0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.339996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296436e-db1a-4624-b23f-146d034903c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.340267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296436e-db1a-4624-b23f-146d034903c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.340938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296436e-db1a-4624-b23f-146d034903c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:19.341244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296436e-db1a-4624-b23f-146d034903c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.342643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296436e-db1a-4624-b23f-146d034903c0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.348529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:19.350518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 63/119] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.356458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432a4f-94cd-4ae4-be76-659db5f0b3ba None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.356744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432a4f-94cd-4ae4-be76-659db5f0b3ba None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.357022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432a4f-94cd-4ae4-be76-659db5f0b3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.357191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df72f7-e898-402f-8498-578e777a18b6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.378311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-26432a4f-94cd-4ae4-be76-659db5f0b3ba None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:19.380407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-26432a4f-94cd-4ae4-be76-659db5f0b3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:19.393617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d82db-7194-4acf-9aa2-5d20a9502f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.394186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d82db-7194-4acf-9aa2-5d20a9502f39 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.394466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d82db-7194-4acf-9aa2-5d20a9502f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.394775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d82db-7194-4acf-9aa2-5d20a9502f39 None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.395882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d82db-7194-4acf-9aa2-5d20a9502f39 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.397338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d82db-7194-4acf-9aa2-5d20a9502f39 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.399014 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 57/120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.404128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d82db-7194-4acf-9aa2-5d20a9502f39 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.462742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 64/121] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:19.462742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26432a4f-94cd-4ae4-be76-659db5f0b3ba None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.484461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:19.524089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d82db-7194-4acf-9aa2-5d20a9502f39 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.570200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.570965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.571333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.571845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.572582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:19.572945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.574400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.575308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26432a4f-94cd-4ae4-be76-659db5f0b3ba None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.582825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 58/122] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.586753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.612409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:19.629377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e8c6-02c7-4880-a88c-02ee5902ca1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.630027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e8c6-02c7-4880-a88c-02ee5902ca1d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.630304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e8c6-02c7-4880-a88c-02ee5902ca1d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.630573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e8c6-02c7-4880-a88c-02ee5902ca1d None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.631747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6e8c6-02c7-4880-a88c-02ee5902ca1d None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.634289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6e8c6-02c7-4880-a88c-02ee5902ca1d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.636247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 65/123] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.639573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e8c6-02c7-4880-a88c-02ee5902ca1d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.700294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91366a-971f-4f12-b4a0-2d7b4f281076 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.736303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845475f-0a69-452c-b927-6762d8511bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.737294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845475f-0a69-452c-b927-6762d8511bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.737806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845475f-0a69-452c-b927-6762d8511bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.738292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845475f-0a69-452c-b927-6762d8511bb6 None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.739671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845475f-0a69-452c-b927-6762d8511bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.741374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845475f-0a69-452c-b927-6762d8511bb6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.743355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 59/124] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.744363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845475f-0a69-452c-b927-6762d8511bb6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.751189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:19.766371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e8c6-02c7-4880-a88c-02ee5902ca1d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.802057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb543f-86e9-47f8-8210-140c657c1ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.802821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb543f-86e9-47f8-8210-140c657c1ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.803097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb543f-86e9-47f8-8210-140c657c1ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.803465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb543f-86e9-47f8-8210-140c657c1ec2 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.804576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb543f-86e9-47f8-8210-140c657c1ec2 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.806254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb543f-86e9-47f8-8210-140c657c1ec2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.810939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 66/125] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.810939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb543f-86e9-47f8-8210-140c657c1ec2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.838449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:19.847837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845475f-0a69-452c-b927-6762d8511bb6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.882559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab61d5-f028-45bd-9c51-15be93daff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.883224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab61d5-f028-45bd-9c51-15be93daff18 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.883592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab61d5-f028-45bd-9c51-15be93daff18 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.883866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab61d5-f028-45bd-9c51-15be93daff18 None admin] PATH_INFO: `/v3/users/13d95fead9ed4b918730e2198000e5e5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.885088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab61d5-f028-45bd-9c51-15be93daff18 None admin] RBAC: Authorizing `identity:get_user(user_id=13d95fead9ed4b918730e2198000e5e5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.887459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab61d5-f028-45bd-9c51-15be93daff18 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.894570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 60/126] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/users/13d95fead9ed4b918730e2198000e5e5 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.894570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab61d5-f028-45bd-9c51-15be93daff18 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.920148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb543f-86e9-47f8-8210-140c657c1ec2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:19.955923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:19.956455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:19.956733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:19.957006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:19.957806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] RBAC: Adding query filter params (role.id=28923fdfbb364d71a285386fd05d9fb2, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf, user.id=63ea915cb96141c684220c4df0efa829) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:19.958162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:19.959652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:19.965435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 67/127] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/role_assignments?user.id=63ea915cb96141c684220c4df0efa829&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=28923fdfbb364d71a285386fd05d9fb2 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:19.966220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:19.997536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:20.012690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab61d5-f028-45bd-9c51-15be93daff18 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.066351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb800e23-fe33-4e01-97d4-171ffa03b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.066351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb800e23-fe33-4e01-97d4-171ffa03b95d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.068813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb800e23-fe33-4e01-97d4-171ffa03b95d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.068813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb800e23-fe33-4e01-97d4-171ffa03b95d None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.070262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb800e23-fe33-4e01-97d4-171ffa03b95d None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:20.072034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb800e23-fe33-4e01-97d4-171ffa03b95d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:20.074672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 61/128] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:20.076014 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb800e23-fe33-4e01-97d4-171ffa03b95d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:20.107816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a633f-a7cd-43e9-8375-437c239610a5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.117684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:20.156181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3603e12-a6a2-477a-84e8-98b163fbedca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.157132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3603e12-a6a2-477a-84e8-98b163fbedca None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.157615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3603e12-a6a2-477a-84e8-98b163fbedca None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.158061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3603e12-a6a2-477a-84e8-98b163fbedca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.175281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3603e12-a6a2-477a-84e8-98b163fbedca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f3603e12-a6a2-477a-84e8-98b163fbedca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:20.182356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:20.183262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:20.184015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:20.186304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 68/129] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:20.187487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3603e12-a6a2-477a-84e8-98b163fbedca None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:20.206819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb800e23-fe33-4e01-97d4-171ffa03b95d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.242381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf8a07-49da-4e5d-830b-96828461581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.243254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf8a07-49da-4e5d-830b-96828461581d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.243656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf8a07-49da-4e5d-830b-96828461581d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.244194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf8a07-49da-4e5d-830b-96828461581d None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/users/0d9b7166b08d4c81a876de0dd637906f/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.246101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf8a07-49da-4e5d-830b-96828461581d None admin] RBAC: Authorizing `identity:create_grant(project_id=5db3d2a60ec94cf1ad027cdaa14910bf, user_id=0d9b7166b08d4c81a876de0dd637906f, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:20.247766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf8a07-49da-4e5d-830b-96828461581d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:20.265327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 62/130] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:20 2022] PUT /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/users/0d9b7166b08d4c81a876de0dd637906f/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:20.266560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf8a07-49da-4e5d-830b-96828461581d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:20.281450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:20.304973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3603e12-a6a2-477a-84e8-98b163fbedca None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.348445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4d948-ffbd-464e-a861-9603bba60009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.349558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4d948-ffbd-464e-a861-9603bba60009 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.350142 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4d948-ffbd-464e-a861-9603bba60009 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.350578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4d948-ffbd-464e-a861-9603bba60009 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.352092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4d948-ffbd-464e-a861-9603bba60009 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:20.353969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4d948-ffbd-464e-a861-9603bba60009 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:20.356078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 69/131] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:20 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:20.357132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4d948-ffbd-464e-a861-9603bba60009 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:20.375947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:20.389065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf8a07-49da-4e5d-830b-96828461581d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.427736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e7161-bcd1-4301-8eba-b0e79d97a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.428442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e7161-bcd1-4301-8eba-b0e79d97a8cc None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.428727 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e7161-bcd1-4301-8eba-b0e79d97a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.429008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e7161-bcd1-4301-8eba-b0e79d97a8cc None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/users/0d9b7166b08d4c81a876de0dd637906f/roles/d06115dbab78488182bcb432bf9bec11` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.431900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e7161-bcd1-4301-8eba-b0e79d97a8cc None admin] RBAC: Authorizing `identity:create_grant(project_id=5db3d2a60ec94cf1ad027cdaa14910bf, user_id=0d9b7166b08d4c81a876de0dd637906f, role_id=d06115dbab78488182bcb432bf9bec11)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:20.433397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e7161-bcd1-4301-8eba-b0e79d97a8cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:20.452059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 63/132] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:20 2022] PUT /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/users/0d9b7166b08d4c81a876de0dd637906f/roles/d06115dbab78488182bcb432bf9bec11 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:20.452859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e7161-bcd1-4301-8eba-b0e79d97a8cc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:20.495721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4d948-ffbd-464e-a861-9603bba60009 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.537804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.538933 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.539448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.540305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.541288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:20.541970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:20.543886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:20.554294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 70/133] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:20.560082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:20.579662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e7161-bcd1-4301-8eba-b0e79d97a8cc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.582482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:20.624633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5ae0e-e423-4618-9ee1-8cc14ae11dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.625563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ae0e-e423-4618-9ee1-8cc14ae11dae None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.625984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ae0e-e423-4618-9ee1-8cc14ae11dae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.626569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ae0e-e423-4618-9ee1-8cc14ae11dae None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/users/13d95fead9ed4b918730e2198000e5e5/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.629034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5ae0e-e423-4618-9ee1-8cc14ae11dae None admin] RBAC: Authorizing `identity:create_grant(project_id=5db3d2a60ec94cf1ad027cdaa14910bf, user_id=13d95fead9ed4b918730e2198000e5e5, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:20.630816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5ae0e-e423-4618-9ee1-8cc14ae11dae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:20.651050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 64/134] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:20 2022] PUT /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/users/13d95fead9ed4b918730e2198000e5e5/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:20.712654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8b5dc-841b-4f19-a77c-253af0fe4e64 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.755531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca3c3f-a814-4bc7-973b-49b6f8a70782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.756110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca3c3f-a814-4bc7-973b-49b6f8a70782 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.756386 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca3c3f-a814-4bc7-973b-49b6f8a70782 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.756664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca3c3f-a814-4bc7-973b-49b6f8a70782 None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.761970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca3c3f-a814-4bc7-973b-49b6f8a70782 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:20.763568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca3c3f-a814-4bc7-973b-49b6f8a70782 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:20.765629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 71/135] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:20 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:20.771569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5ae0e-e423-4618-9ee1-8cc14ae11dae None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:20.788406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:20.910904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5ae0e-e423-4618-9ee1-8cc14ae11dae None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:20.947201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2dc14-f209-4369-afa4-af43041c7276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:20.947859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2dc14-f209-4369-afa4-af43041c7276 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.948142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2dc14-f209-4369-afa4-af43041c7276 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.948417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2dc14-f209-4369-afa4-af43041c7276 None admin] PATH_INFO: `/v3/projects/57244b4e47524ab28ec4382ca7fb13a9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.949575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2dc14-f209-4369-afa4-af43041c7276 None admin] RBAC: Authorizing `identity:get_project(project_id=57244b4e47524ab28ec4382ca7fb13a9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:20.951071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2dc14-f209-4369-afa4-af43041c7276 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:20.952733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 65/136] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:20 2022] GET /identity/v3/projects/57244b4e47524ab28ec4382ca7fb13a9 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:20.958761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca3c3f-a814-4bc7-973b-49b6f8a70782 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:20.995505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3d700-2075-4a07-b449-4b1399934d4a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:20.995857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3d700-2075-4a07-b449-4b1399934d4a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:20.996140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3d700-2075-4a07-b449-4b1399934d4a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:20.997869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 66/137] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:21.004650 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9e54d-5c57-475a-a85d-4d8b1ea387a6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:21.004949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9e54d-5c57-475a-a85d-4d8b1ea387a6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:21.005225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9e54d-5c57-475a-a85d-4d8b1ea387a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:21.025337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-43f9e54d-5c57-475a-a85d-4d8b1ea387a6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:21.026502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-43f9e54d-5c57-475a-a85d-4d8b1ea387a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:21.102839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 67/138] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:21.124313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:21.162344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca3c3f-a814-4bc7-973b-49b6f8a70782 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:21.201572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731d2d5-b675-4ce8-9961-a2dbb389ebce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:21.202247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731d2d5-b675-4ce8-9961-a2dbb389ebce None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:21.202528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731d2d5-b675-4ce8-9961-a2dbb389ebce None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:21.202830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731d2d5-b675-4ce8-9961-a2dbb389ebce None admin] PATH_INFO: `/v3/projects/57244b4e47524ab28ec4382ca7fb13a9/users/0d9b7166b08d4c81a876de0dd637906f/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:21.204810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731d2d5-b675-4ce8-9961-a2dbb389ebce None admin] RBAC: Authorizing `identity:create_grant(project_id=57244b4e47524ab28ec4382ca7fb13a9, user_id=0d9b7166b08d4c81a876de0dd637906f, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:21.206323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731d2d5-b675-4ce8-9961-a2dbb389ebce None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:21.221645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 72/139] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:20 2022] PUT /identity/v3/projects/57244b4e47524ab28ec4382ca7fb13a9/users/0d9b7166b08d4c81a876de0dd637906f/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:21.460343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9e54d-5c57-475a-a85d-4d8b1ea387a6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:21.467830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:21.565509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9e54d-5c57-475a-a85d-4d8b1ea387a6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:21.605523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3187e1f-d6da-406a-983f-cd2ef6975fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:21.606552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3187e1f-d6da-406a-983f-cd2ef6975fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:21.607108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3187e1f-d6da-406a-983f-cd2ef6975fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:21.607566 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3187e1f-d6da-406a-983f-cd2ef6975fc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:21.611378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3187e1f-d6da-406a-983f-cd2ef6975fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:21.613036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3187e1f-d6da-406a-983f-cd2ef6975fc7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:21.614865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 68/140] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:21.620844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731d2d5-b675-4ce8-9961-a2dbb389ebce None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:21.751739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731d2d5-b675-4ce8-9961-a2dbb389ebce None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:21.786474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d567bc3-e2b3-4e55-9ab5-9cd9bf1993f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:21.787442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567bc3-e2b3-4e55-9ab5-9cd9bf1993f7 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:21.787936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567bc3-e2b3-4e55-9ab5-9cd9bf1993f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:21.788389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567bc3-e2b3-4e55-9ab5-9cd9bf1993f7 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:21.789226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d567bc3-e2b3-4e55-9ab5-9cd9bf1993f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:21.790992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d567bc3-e2b3-4e55-9ab5-9cd9bf1993f7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:21.808727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:21.821524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 73/141] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:21:21 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:22.974595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedde28-4fad-44d5-bc3f-711ecc90581d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:22.975135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedde28-4fad-44d5-bc3f-711ecc90581d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:22.975209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedde28-4fad-44d5-bc3f-711ecc90581d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:22.976785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 69/142] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:22.982228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1dc36-c212-42b3-97c6-7d6d928ce376 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:22.982684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1dc36-c212-42b3-97c6-7d6d928ce376 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:22.982890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1dc36-c212-42b3-97c6-7d6d928ce376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.002805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-97b1dc36-c212-42b3-97c6-7d6d928ce376 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:23.003997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97b1dc36-c212-42b3-97c6-7d6d928ce376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:23.074334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 74/143] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:23.257432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3c933-6391-4040-9790-8f3c0cf3fb5d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:23.257685 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3c933-6391-4040-9790-8f3c0cf3fb5d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:23.257925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3c933-6391-4040-9790-8f3c0cf3fb5d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.259663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 70/144] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:23.265064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f70-6467-4960-a99d-e718be120297 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:23.265352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f70-6467-4960-a99d-e718be120297 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:23.265630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f70-6467-4960-a99d-e718be120297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.287125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5dee4f70-6467-4960-a99d-e718be120297 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:23.288289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5dee4f70-6467-4960-a99d-e718be120297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:23.366899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 75/145] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:23.398757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77ed2a-57b8-489f-9f9f-e4adbd6cd0ca None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:23.399094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77ed2a-57b8-489f-9f9f-e4adbd6cd0ca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:23.399421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77ed2a-57b8-489f-9f9f-e4adbd6cd0ca None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.400953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 71/146] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:23.406113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a953d-5114-4489-943b-3a0c5318a994 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:23.406407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a953d-5114-4489-943b-3a0c5318a994 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:23.406744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a953d-5114-4489-943b-3a0c5318a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.432009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-620a953d-5114-4489-943b-3a0c5318a994 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:23.433318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-620a953d-5114-4489-943b-3a0c5318a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:23.477344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77ed2a-57b8-489f-9f9f-e4adbd6cd0ca None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:23.496604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:23.517905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 76/147] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:23.519022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076dc27-3146-49f7-8aaf-6c1ef100a091 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:23.519310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076dc27-3146-49f7-8aaf-6c1ef100a091 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:23.519581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076dc27-3146-49f7-8aaf-6c1ef100a091 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.520995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 77/148] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:23.527649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f0cef-ac56-445b-ae4d-cfa0ac5f1c1a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:23.527943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f0cef-ac56-445b-ae4d-cfa0ac5f1c1a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:23.528592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f0cef-ac56-445b-ae4d-cfa0ac5f1c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.548925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c45f0cef-ac56-445b-ae4d-cfa0ac5f1c1a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:23.550673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c45f0cef-ac56-445b-ae4d-cfa0ac5f1c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:23.622411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 78/149] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:23.640862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:23.680386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77ed2a-57b8-489f-9f9f-e4adbd6cd0ca None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:23.722133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95cf05-a81b-424c-b0a7-7ca85a04f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:23.722811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95cf05-a81b-424c-b0a7-7ca85a04f899 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:23.723095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95cf05-a81b-424c-b0a7-7ca85a04f899 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:23.723369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95cf05-a81b-424c-b0a7-7ca85a04f899 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.740030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95cf05-a81b-424c-b0a7-7ca85a04f899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-eb95cf05-a81b-424c-b0a7-7ca85a04f899 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:23.745396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:23.746243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:23.747086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 72/150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:23.752595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f0cef-ac56-445b-ae4d-cfa0ac5f1c1a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:23.873190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f0cef-ac56-445b-ae4d-cfa0ac5f1c1a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:23.892269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:23.914243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:23.915052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:23.915334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:23.915604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:23.916286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:23.916584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:23.917978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:23.927021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 79/151] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:23.932612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95cf05-a81b-424c-b0a7-7ca85a04f899 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:23.953311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:24.138036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95cf05-a81b-424c-b0a7-7ca85a04f899 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:24.141950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33533703-6179-4806-8c27-8291a1a196e1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:24.204357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cf1e-a39d-4fac-8f14-354ab7fb6dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:24.205011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cf1e-a39d-4fac-8f14-354ab7fb6dea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:24.205287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cf1e-a39d-4fac-8f14-354ab7fb6dea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:24.205556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cf1e-a39d-4fac-8f14-354ab7fb6dea None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:24.209918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002cf1e-a39d-4fac-8f14-354ab7fb6dea None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:24.211949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002cf1e-a39d-4fac-8f14-354ab7fb6dea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:24.213671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 73/152] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:23 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:24.214448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cf1e-a39d-4fac-8f14-354ab7fb6dea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:24.221400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:24.233954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b589d-2aa6-4ebf-8a01-fc4b24a4e253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:24.234947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b589d-2aa6-4ebf-8a01-fc4b24a4e253 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:24.235368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b589d-2aa6-4ebf-8a01-fc4b24a4e253 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:24.235774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b589d-2aa6-4ebf-8a01-fc4b24a4e253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:24.261493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b589d-2aa6-4ebf-8a01-fc4b24a4e253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e04b589d-2aa6-4ebf-8a01-fc4b24a4e253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:24.268330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:24.269172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:24.269982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 80/153] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:24.270624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b589d-2aa6-4ebf-8a01-fc4b24a4e253 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:24.358767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:24.450556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cf1e-a39d-4fac-8f14-354ab7fb6dea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:24.486442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b589d-2aa6-4ebf-8a01-fc4b24a4e253 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:24.538779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abcbe1-4ab3-420b-bba3-7e81b7e84087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:24.539483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abcbe1-4ab3-420b-bba3-7e81b7e84087 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:24.539764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abcbe1-4ab3-420b-bba3-7e81b7e84087 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:24.540206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abcbe1-4ab3-420b-bba3-7e81b7e84087 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:24.561408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abcbe1-4ab3-420b-bba3-7e81b7e84087 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:24.565601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96f850-c3cb-455f-a22f-9c867790b40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:24.566526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96f850-c3cb-455f-a22f-9c867790b40b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:24.566967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96f850-c3cb-455f-a22f-9c867790b40b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:24.567368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96f850-c3cb-455f-a22f-9c867790b40b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-42abcbe1-4ab3-420b-bba3-7e81b7e84087 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:24.571025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:24.572073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:24.572820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 74/154] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:24.573629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abcbe1-4ab3-420b-bba3-7e81b7e84087 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:24.597748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96f850-c3cb-455f-a22f-9c867790b40b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-fc96f850-c3cb-455f-a22f-9c867790b40b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:24.603677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:24.604598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:24.605344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 81/155] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:24.605969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96f850-c3cb-455f-a22f-9c867790b40b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:24.745077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:24.773237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:24.783904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96f850-c3cb-455f-a22f-9c867790b40b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:24.826424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abcbe1-4ab3-420b-bba3-7e81b7e84087 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:24.835422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:24.836055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:24.836331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:24.836607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:24.837356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:24.837757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:24.839299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:24.855251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 82/156] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:24.856730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:24.905964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5f7b5-afd6-4ccb-8f1e-192329d93271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:24.906970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f7b5-afd6-4ccb-8f1e-192329d93271 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:24.907398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f7b5-afd6-4ccb-8f1e-192329d93271 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:24.907806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f7b5-afd6-4ccb-8f1e-192329d93271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.030301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ce35f-7ee5-48ab-8a93-ba9ec01e8079 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:25.043910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5f7b5-afd6-4ccb-8f1e-192329d93271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:25.045726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5f7b5-afd6-4ccb-8f1e-192329d93271 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:25.095028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:25.095657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.095933 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.096206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.097170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:25.097603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:25.099274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:25.109071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 83/157] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:25.109908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:25.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:25.159895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:25.239337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 75/158] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:25.242224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5f7b5-afd6-4ccb-8f1e-192329d93271 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:25.302344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c7516-5a4d-464b-bb0e-a742b0280c16 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:25.368805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e20fed-6796-48b0-b4b1-40de6b1aed3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:25.369533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e20fed-6796-48b0-b4b1-40de6b1aed3e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.369810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e20fed-6796-48b0-b4b1-40de6b1aed3e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.370080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e20fed-6796-48b0-b4b1-40de6b1aed3e None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.370779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e20fed-6796-48b0-b4b1-40de6b1aed3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:25.371081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e20fed-6796-48b0-b4b1-40de6b1aed3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:25.372472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e20fed-6796-48b0-b4b1-40de6b1aed3e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:25.381164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 84/159] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:25.386950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4eea-e680-4c44-90d1-5283796e44b6 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.387233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4eea-e680-4c44-90d1-5283796e44b6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.387508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d4eea-e680-4c44-90d1-5283796e44b6 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.389136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 85/160] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:25.389867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4eea-e680-4c44-90d1-5283796e44b6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:25.438317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5f7b5-afd6-4ccb-8f1e-192329d93271 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:25.513472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fe616-9295-4421-b4b9-eda820ea422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:25.514115 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fe616-9295-4421-b4b9-eda820ea422c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.514390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fe616-9295-4421-b4b9-eda820ea422c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.514661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fe616-9295-4421-b4b9-eda820ea422c None admin] PATH_INFO: `/v3/users/13d95fead9ed4b918730e2198000e5e5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.517423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fe616-9295-4421-b4b9-eda820ea422c None admin] RBAC: Authorizing `identity:get_user(user_id=13d95fead9ed4b918730e2198000e5e5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:25.518925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fe616-9295-4421-b4b9-eda820ea422c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:25.525175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 76/161] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:25 2022] GET /identity/v3/users/13d95fead9ed4b918730e2198000e5e5 => generated 247 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:25.526079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fe616-9295-4421-b4b9-eda820ea422c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:25.543099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:25.565874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:25.568913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d4eea-e680-4c44-90d1-5283796e44b6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:25.611141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa1d47-4030-4b22-a109-e8eb15f67c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:25.614738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa1d47-4030-4b22-a109-e8eb15f67c9a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.614738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa1d47-4030-4b22-a109-e8eb15f67c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.614738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa1d47-4030-4b22-a109-e8eb15f67c9a None admin] PATH_INFO: `/v3/users/63ea915cb96141c684220c4df0efa829` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.614738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa1d47-4030-4b22-a109-e8eb15f67c9a None admin] RBAC: Authorizing `identity:get_user(user_id=63ea915cb96141c684220c4df0efa829)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:25.615579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa1d47-4030-4b22-a109-e8eb15f67c9a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:25.618146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 86/162] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:25 2022] GET /identity/v3/users/63ea915cb96141c684220c4df0efa829 => generated 289 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:25.619934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa1d47-4030-4b22-a109-e8eb15f67c9a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:25.644833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fe616-9295-4421-b4b9-eda820ea422c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:25.688401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74829b4f-0b30-4053-8d59-db89b4ab3e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:25.689336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829b4f-0b30-4053-8d59-db89b4ab3e72 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.689628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829b4f-0b30-4053-8d59-db89b4ab3e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.690521 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829b4f-0b30-4053-8d59-db89b4ab3e72 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.691694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74829b4f-0b30-4053-8d59-db89b4ab3e72 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:25.693323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74829b4f-0b30-4053-8d59-db89b4ab3e72 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:25.695767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 77/163] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:25 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:25.697252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c341-5c3b-47d5-bdde-6b40ec121533 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.698363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c341-5c3b-47d5-bdde-6b40ec121533 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.698652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c341-5c3b-47d5-bdde-6b40ec121533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.720243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5e75c341-5c3b-47d5-bdde-6b40ec121533 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:25.722019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e75c341-5c3b-47d5-bdde-6b40ec121533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:25.730209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa1d47-4030-4b22-a109-e8eb15f67c9a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:25.767240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca6eff-4afd-4b9a-94d7-465ffdd89e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:25.767885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6eff-4afd-4b9a-94d7-465ffdd89e90 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.768163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6eff-4afd-4b9a-94d7-465ffdd89e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.768515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6eff-4afd-4b9a-94d7-465ffdd89e90 None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.769583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca6eff-4afd-4b9a-94d7-465ffdd89e90 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:25.771259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca6eff-4afd-4b9a-94d7-465ffdd89e90 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:25.772876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 87/164] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:25 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:25.773646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca6eff-4afd-4b9a-94d7-465ffdd89e90 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:25.796480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:25.809030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 78/165] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:25.811022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444418f7-a0b3-4afb-a6f2-0b1a37aa8707 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.811311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444418f7-a0b3-4afb-a6f2-0b1a37aa8707 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.811584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444418f7-a0b3-4afb-a6f2-0b1a37aa8707 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.812999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 79/166] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:25.817632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444418f7-a0b3-4afb-a6f2-0b1a37aa8707 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:25.847943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:25.895614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca6eff-4afd-4b9a-94d7-465ffdd89e90 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:25.940844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444418f7-a0b3-4afb-a6f2-0b1a37aa8707 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:25.943550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914394-b087-4be8-8a8f-1258b46af288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:25.944256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914394-b087-4be8-8a8f-1258b46af288 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.945460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914394-b087-4be8-8a8f-1258b46af288 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.945746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914394-b087-4be8-8a8f-1258b46af288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.979067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d912960-595b-4965-a5c9-31bceb65995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:25.980100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912960-595b-4965-a5c9-31bceb65995a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:25.980578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912960-595b-4965-a5c9-31bceb65995a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:25.981039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d912960-595b-4965-a5c9-31bceb65995a None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:25.982408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d912960-595b-4965-a5c9-31bceb65995a None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:25.984089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d912960-595b-4965-a5c9-31bceb65995a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:25.986016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 80/167] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:25 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:25.987002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d912960-595b-4965-a5c9-31bceb65995a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:26.036733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a914394-b087-4be8-8a8f-1258b46af288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:26.038501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a914394-b087-4be8-8a8f-1258b46af288 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:26.063513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:26.092394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:26.100567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d912960-595b-4965-a5c9-31bceb65995a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:26.136962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:26.137672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.137952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.138223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.138993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] RBAC: Adding query filter params (role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf, user.id=0d9b7166b08d4c81a876de0dd637906f) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:26.139298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:26.140762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:26.146549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 81/168] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:25 2022] GET /identity/v3/role_assignments?user.id=0d9b7166b08d4c81a876de0dd637906f&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=5bbcda16701c451f9419aefc4671840e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:26.150973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:26.168231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 88/169] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:26.170656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5f85-929c-4265-96a6-a2c4f335e83a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.170979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5f85-929c-4265-96a6-a2c4f335e83a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.171253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5f85-929c-4265-96a6-a2c4f335e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.192339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1a7a5f85-929c-4265-96a6-a2c4f335e83a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:26.193570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1a7a5f85-929c-4265-96a6-a2c4f335e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:26.272297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5881c3-8f04-451f-ae53-5438939393fe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:26.290910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 89/170] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:26.292436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a5f85-929c-4265-96a6-a2c4f335e83a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:26.310499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c506e2-e9a6-4f62-be05-125512812a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:26.311216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c506e2-e9a6-4f62-be05-125512812a75 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.311473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c506e2-e9a6-4f62-be05-125512812a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.313033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c506e2-e9a6-4f62-be05-125512812a75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.357733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:26.365723 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:26.412198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c506e2-e9a6-4f62-be05-125512812a75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:26.413684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c506e2-e9a6-4f62-be05-125512812a75 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:26.428394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a5f85-929c-4265-96a6-a2c4f335e83a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:26.464252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68fc8a-c170-4b73-ae33-ab40499fe562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:26.465016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68fc8a-c170-4b73-ae33-ab40499fe562 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.465289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68fc8a-c170-4b73-ae33-ab40499fe562 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.465554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68fc8a-c170-4b73-ae33-ab40499fe562 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/users/63ea915cb96141c684220c4df0efa829/roles/28923fdfbb364d71a285386fd05d9fb2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.467435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68fc8a-c170-4b73-ae33-ab40499fe562 None admin] RBAC: Authorizing `identity:create_grant(project_id=5db3d2a60ec94cf1ad027cdaa14910bf, user_id=63ea915cb96141c684220c4df0efa829, role_id=28923fdfbb364d71a285386fd05d9fb2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:26.468909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68fc8a-c170-4b73-ae33-ab40499fe562 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:26.483269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 90/171] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:26 2022] PUT /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/users/63ea915cb96141c684220c4df0efa829/roles/28923fdfbb364d71a285386fd05d9fb2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:26.484496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c860afc-e163-4d5e-a47a-9977d8e679d3 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.484779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c860afc-e163-4d5e-a47a-9977d8e679d3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.485017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c860afc-e163-4d5e-a47a-9977d8e679d3 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.486414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 91/172] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:26.487528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff0473-81d3-4935-8196-6e48f73b81be None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.487807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff0473-81d3-4935-8196-6e48f73b81be None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.488073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff0473-81d3-4935-8196-6e48f73b81be None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.489324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 92/173] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:26.494244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff0473-81d3-4935-8196-6e48f73b81be None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:26.541460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 82/174] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:26.541460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c506e2-e9a6-4f62-be05-125512812a75 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:26.605590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:26.629894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff0473-81d3-4935-8196-6e48f73b81be None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:26.643215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:26.651877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c506e2-e9a6-4f62-be05-125512812a75 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:26.673756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de69cac-daef-4bac-9cab-dbbbc3d77028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:26.674930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de69cac-daef-4bac-9cab-dbbbc3d77028 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.675360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de69cac-daef-4bac-9cab-dbbbc3d77028 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.675634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de69cac-daef-4bac-9cab-dbbbc3d77028 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.685874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de69cac-daef-4bac-9cab-dbbbc3d77028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:26.688650 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba8b10-9d9c-42bb-b6e4-63c1c1923a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:26.689354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba8b10-9d9c-42bb-b6e4-63c1c1923a27 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.689634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba8b10-9d9c-42bb-b6e4-63c1c1923a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.689904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba8b10-9d9c-42bb-b6e4-63c1c1923a27 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8de69cac-daef-4bac-9cab-dbbbc3d77028 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:26.691139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:26.692016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:26.692797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:26.693381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba8b10-9d9c-42bb-b6e4-63c1c1923a27 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:26.693448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 93/175] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:26.693518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba8b10-9d9c-42bb-b6e4-63c1c1923a27 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:26.693584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d27fc-eacd-4e42-a8e2-ee220658b758 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.694151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d27fc-eacd-4e42-a8e2-ee220658b758 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.694151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d27fc-eacd-4e42-a8e2-ee220658b758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.698871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 83/176] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:26 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:26.698871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc74ff-85a0-41a5-a62a-f40fb5d2b52e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:26.698871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc74ff-85a0-41a5-a62a-f40fb5d2b52e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:26.698871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc74ff-85a0-41a5-a62a-f40fb5d2b52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:26.714430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4d1d27fc-eacd-4e42-a8e2-ee220658b758 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:26.715634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4d1d27fc-eacd-4e42-a8e2-ee220658b758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:26.716789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-29dc74ff-85a0-41a5-a62a-f40fb5d2b52e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:26.717943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-29dc74ff-85a0-41a5-a62a-f40fb5d2b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:26.787281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 94/177] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:26.795115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d27fc-eacd-4e42-a8e2-ee220658b758 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:26.795662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 84/178] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:26.798225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc74ff-85a0-41a5-a62a-f40fb5d2b52e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:26.903060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:26.972886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:26.976945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc74ff-85a0-41a5-a62a-f40fb5d2b52e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:26.980778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d27fc-eacd-4e42-a8e2-ee220658b758 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.043038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d586e0-c5b0-48d2-99ac-2ae4f482dcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.043687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d586e0-c5b0-48d2-99ac-2ae4f482dcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.043961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d586e0-c5b0-48d2-99ac-2ae4f482dcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.044231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d586e0-c5b0-48d2-99ac-2ae4f482dcb2 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.045336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29150eaf-51d4-428b-8949-94da260be253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.045967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29150eaf-51d4-428b-8949-94da260be253 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.046239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29150eaf-51d4-428b-8949-94da260be253 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.046507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29150eaf-51d4-428b-8949-94da260be253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.047584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d586e0-c5b0-48d2-99ac-2ae4f482dcb2 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.049057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d586e0-c5b0-48d2-99ac-2ae4f482dcb2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.050698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 95/179] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:26 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.051485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d586e0-c5b0-48d2-99ac-2ae4f482dcb2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.075734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29150eaf-51d4-428b-8949-94da260be253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-29150eaf-51d4-428b-8949-94da260be253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:27.080619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:27.081434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:27.082192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:27.083895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 85/180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:27.084687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29150eaf-51d4-428b-8949-94da260be253 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.180629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d586e0-c5b0-48d2-99ac-2ae4f482dcb2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.193462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29150eaf-51d4-428b-8949-94da260be253 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.210819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:27.218035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.218655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.218959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.219298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.220013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:27.220366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.221923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.227941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.228542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.228814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.229076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.229796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf, user.id=13d95fead9ed4b918730e2198000e5e5) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:27.230095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.230239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 96/181] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.234868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.234960 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.241470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 86/182] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/role_assignments?user.id=13d95fead9ed4b918730e2198000e5e5&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=241e99d365564c0fa95cb712f4004785 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.250887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.273517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:27.370882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346af8d1-f3cc-4ef3-9092-3e32e7f8afa8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.387548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f31b4c-3619-4dcb-b482-769c7f409904 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.408886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.409518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.409793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.410060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.410830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] RBAC: Adding query filter params (role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf, user.id=0d9b7166b08d4c81a876de0dd637906f) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:27.411188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.413356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.420168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 97/183] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/role_assignments?user.id=0d9b7166b08d4c81a876de0dd637906f&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=d06115dbab78488182bcb432bf9bec11 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.420816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.442581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0d5eb-5df4-4734-b570-ab0aea289b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.443191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0d5eb-5df4-4734-b570-ab0aea289b27 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.443467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0d5eb-5df4-4734-b570-ab0aea289b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.443737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0d5eb-5df4-4734-b570-ab0aea289b27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.462098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0d5eb-5df4-4734-b570-ab0aea289b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d1e0d5eb-5df4-4734-b570-ab0aea289b27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:27.469370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:27.470327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:27.471132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 87/184] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:27.471783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0d5eb-5df4-4734-b570-ab0aea289b27 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.529938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:27.555844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:27.569437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605881a-c4c7-4b17-9668-0db294d095f1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.609366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0d5eb-5df4-4734-b570-ab0aea289b27 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.620878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.621549 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.621830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.622098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.622921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:27.623222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.624655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.636879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 98/185] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.637529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.660077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479e363-b422-43a0-b6a2-f83c4a2f6a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.660721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479e363-b422-43a0-b6a2-f83c4a2f6a46 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.660995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479e363-b422-43a0-b6a2-f83c4a2f6a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.661269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479e363-b422-43a0-b6a2-f83c4a2f6a46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.662414 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479e363-b422-43a0-b6a2-f83c4a2f6a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.664807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479e363-b422-43a0-b6a2-f83c4a2f6a46 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.665561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 88/186] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.666524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479e363-b422-43a0-b6a2-f83c4a2f6a46 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.763097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f56b7-92ae-49cc-8d91-84ba77b3441d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.787375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479e363-b422-43a0-b6a2-f83c4a2f6a46 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.798153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:27.798751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1ed1-e25d-489b-851b-4243d2c93d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.799690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1ed1-e25d-489b-851b-4243d2c93d65 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.799953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1ed1-e25d-489b-851b-4243d2c93d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.800243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1ed1-e25d-489b-851b-4243d2c93d65 None admin] PATH_INFO: `/v3/users/0d9b7166b08d4c81a876de0dd637906f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.801320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f1ed1-e25d-489b-851b-4243d2c93d65 None admin] RBAC: Authorizing `identity:get_user(user_id=0d9b7166b08d4c81a876de0dd637906f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.802806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f1ed1-e25d-489b-851b-4243d2c93d65 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.804553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 99/187] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/users/0d9b7166b08d4c81a876de0dd637906f => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.808171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1ed1-e25d-489b-851b-4243d2c93d65 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.824875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.825494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.825785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.826036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.826763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:27.827045 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.828423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.836214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 89/188] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.837058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.841390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:27.922107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1ed1-e25d-489b-851b-4243d2c93d65 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.942246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8118e-e33a-4082-9afd-2bc70e90a517 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:27.958053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fbff8-1f60-401f-b400-7a0fdf5a70cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.958695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fbff8-1f60-401f-b400-7a0fdf5a70cf None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.959018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fbff8-1f60-401f-b400-7a0fdf5a70cf None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.959291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fbff8-1f60-401f-b400-7a0fdf5a70cf None admin] PATH_INFO: `/v3/users/5556de0aed2b4bbb95048e86a39f24fd` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.974184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fbff8-1f60-401f-b400-7a0fdf5a70cf None admin] RBAC: Authorizing `identity:get_user(user_id=5556de0aed2b4bbb95048e86a39f24fd)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.975760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fbff8-1f60-401f-b400-7a0fdf5a70cf None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:27.976848 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e9b35-f450-40e8-bd3f-30704d48005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:27.977610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 100/189] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/users/5556de0aed2b4bbb95048e86a39f24fd => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:27.978242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fbff8-1f60-401f-b400-7a0fdf5a70cf None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:27.982384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e9b35-f450-40e8-bd3f-30704d48005b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:27.982753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e9b35-f450-40e8-bd3f-30704d48005b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:27.983040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e9b35-f450-40e8-bd3f-30704d48005b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:27.983722 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e9b35-f450-40e8-bd3f-30704d48005b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:27.985172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e9b35-f450-40e8-bd3f-30704d48005b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:28.014882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 90/190] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:21:27 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:28.014882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e9b35-f450-40e8-bd3f-30704d48005b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:28.062759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:28.097734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:28.126256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fbff8-1f60-401f-b400-7a0fdf5a70cf None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:28.157183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e9b35-f450-40e8-bd3f-30704d48005b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:28.176662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecbddc-0923-4b48-9762-a2681c9a26b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:28.177585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecbddc-0923-4b48-9762-a2681c9a26b4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.177874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecbddc-0923-4b48-9762-a2681c9a26b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.178140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecbddc-0923-4b48-9762-a2681c9a26b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.200783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c98acb-0848-44b4-9dbc-071b4af15190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:28.202476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c98acb-0848-44b4-9dbc-071b4af15190 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.202476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c98acb-0848-44b4-9dbc-071b4af15190 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.202476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c98acb-0848-44b4-9dbc-071b4af15190 None admin] PATH_INFO: `/v3/users/5556de0aed2b4bbb95048e86a39f24fd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.204092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c98acb-0848-44b4-9dbc-071b4af15190 None admin] RBAC: Authorizing `identity:get_user(user_id=5556de0aed2b4bbb95048e86a39f24fd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:28.206094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c98acb-0848-44b4-9dbc-071b4af15190 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:28.206674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 91/191] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:28 2022] GET /identity/v3/users/5556de0aed2b4bbb95048e86a39f24fd => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:28.207507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c98acb-0848-44b4-9dbc-071b4af15190 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:28.265693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecbddc-0923-4b48-9762-a2681c9a26b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:28.267370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecbddc-0923-4b48-9762-a2681c9a26b4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:28.330349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:28.334618 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c98acb-0848-44b4-9dbc-071b4af15190 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:28.376363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:28.391513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b0754-7fd4-41d9-9a38-6fe0d93a9a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:28.392713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b0754-7fd4-41d9-9a38-6fe0d93a9a1b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.393131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b0754-7fd4-41d9-9a38-6fe0d93a9a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.393563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b0754-7fd4-41d9-9a38-6fe0d93a9a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.403392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 101/192] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:28.404748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecbddc-0923-4b48-9762-a2681c9a26b4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:28.480193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b0754-7fd4-41d9-9a38-6fe0d93a9a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:28.482572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b0754-7fd4-41d9-9a38-6fe0d93a9a1b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:28.536410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecbddc-0923-4b48-9762-a2681c9a26b4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:28.585021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7209a-b3e0-410c-beb9-baa37ab44e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:28.586415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7209a-b3e0-410c-beb9-baa37ab44e10 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.587247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7209a-b3e0-410c-beb9-baa37ab44e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.587739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7209a-b3e0-410c-beb9-baa37ab44e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.599041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:28.610901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 92/193] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:28.610901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b0754-7fd4-41d9-9a38-6fe0d93a9a1b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:28.666211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:28.690104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7209a-b3e0-410c-beb9-baa37ab44e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:28.692281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7209a-b3e0-410c-beb9-baa37ab44e10 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:28.722285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b0754-7fd4-41d9-9a38-6fe0d93a9a1b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:28.759153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad4f2e-867c-4ab3-88d4-2ed4579eea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:28.759679 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad4f2e-867c-4ab3-88d4-2ed4579eea92 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.759958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad4f2e-867c-4ab3-88d4-2ed4579eea92 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.760225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad4f2e-867c-4ab3-88d4-2ed4579eea92 None admin] PATH_INFO: `/v3/projects/57244b4e47524ab28ec4382ca7fb13a9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.761377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad4f2e-867c-4ab3-88d4-2ed4579eea92 None admin] RBAC: Authorizing `identity:get_project(project_id=57244b4e47524ab28ec4382ca7fb13a9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:28.763341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad4f2e-867c-4ab3-88d4-2ed4579eea92 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:28.769758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 93/194] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:28 2022] GET /identity/v3/projects/57244b4e47524ab28ec4382ca7fb13a9 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:28.769758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a3972-be4e-47a4-9a33-0607aa597d06 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.769758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a3972-be4e-47a4-9a33-0607aa597d06 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.769758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a3972-be4e-47a4-9a33-0607aa597d06 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.769758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 94/195] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:28.770281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a3972-be4e-47a4-9a33-0607aa597d06 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:28.819048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 102/196] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:28.820220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7209a-b3e0-410c-beb9-baa37ab44e10 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:28.848271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:28.896104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a3972-be4e-47a4-9a33-0607aa597d06 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:28.916327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:28.938658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7209a-b3e0-410c-beb9-baa37ab44e10 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:28.938962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ef114-797a-4cda-a1a2-ad1ce1c4e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:28.939749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef114-797a-4cda-a1a2-ad1ce1c4e209 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.940155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef114-797a-4cda-a1a2-ad1ce1c4e209 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.941091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef114-797a-4cda-a1a2-ad1ce1c4e209 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.943127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ef114-797a-4cda-a1a2-ad1ce1c4e209 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:28.944720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ef114-797a-4cda-a1a2-ad1ce1c4e209 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:28.946554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 95/197] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:28 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:28.947999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8343347-89e7-4972-a4e5-0cde02cd4e27 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.948425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8343347-89e7-4972-a4e5-0cde02cd4e27 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.948810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8343347-89e7-4972-a4e5-0cde02cd4e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.970540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e8343347-89e7-4972-a4e5-0cde02cd4e27 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:28.972200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e8343347-89e7-4972-a4e5-0cde02cd4e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:28.981620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:28.982583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:28.982841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:28.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:28.984055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] RBAC: Adding query filter params (role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=57244b4e47524ab28ec4382ca7fb13a9, user.id=0d9b7166b08d4c81a876de0dd637906f) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:28.984454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:28.986240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:28.994860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 103/198] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:28 2022] GET /identity/v3/role_assignments?user.id=0d9b7166b08d4c81a876de0dd637906f&scope.project.id=57244b4e47524ab28ec4382ca7fb13a9&role.id=5bbcda16701c451f9419aefc4671840e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:28.995865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:29.056460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 96/199] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:29.058074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8343347-89e7-4972-a4e5-0cde02cd4e27 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:29.135476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c3637-199f-4278-bef7-ddc35e6a0406 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:29.163674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:29.189018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69d18-12f7-4a87-9e37-6c58fcf7b754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:29.190966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69d18-12f7-4a87-9e37-6c58fcf7b754 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:29.191247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69d18-12f7-4a87-9e37-6c58fcf7b754 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:29.191517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69d18-12f7-4a87-9e37-6c58fcf7b754 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:29.193026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e69d18-12f7-4a87-9e37-6c58fcf7b754 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:29.194276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e69d18-12f7-4a87-9e37-6c58fcf7b754 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:29.201623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 104/200] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:28 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:29.208034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69d18-12f7-4a87-9e37-6c58fcf7b754 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:29.249392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:29.266369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8343347-89e7-4972-a4e5-0cde02cd4e27 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:29.345620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:29.346172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:29.346446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:29.346711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:29.347478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=a33c93942a544564b55765288214ed83, user.id=5556de0aed2b4bbb95048e86a39f24fd) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:29.347779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:29.349233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:29.362350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 97/201] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:29 2022] GET /identity/v3/role_assignments?user.id=5556de0aed2b4bbb95048e86a39f24fd&scope.project.id=a33c93942a544564b55765288214ed83&role.id=241e99d365564c0fa95cb712f4004785 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:29.370774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69d18-12f7-4a87-9e37-6c58fcf7b754 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:29.409443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:29.464107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:29.464747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:29.465030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:29.465296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:29.466047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] RBAC: Adding query filter params (role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=a33c93942a544564b55765288214ed83, user.id=5556de0aed2b4bbb95048e86a39f24fd) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:29.466358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:29.467862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:29.485074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 105/202] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:29 2022] GET /identity/v3/role_assignments?user.id=5556de0aed2b4bbb95048e86a39f24fd&scope.project.id=a33c93942a544564b55765288214ed83&role.id=d06115dbab78488182bcb432bf9bec11 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:29.528234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63e300-a5b8-4e81-b5a7-f3c7fe8fc762 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:29.558290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:29.569186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7f73a-a55c-4457-a721-3be455ea66fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:29.570197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7f73a-a55c-4457-a721-3be455ea66fe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:29.570696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7f73a-a55c-4457-a721-3be455ea66fe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:29.571189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7f73a-a55c-4457-a721-3be455ea66fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:29.590039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7f73a-a55c-4457-a721-3be455ea66fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-54b7f73a-a55c-4457-a721-3be455ea66fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:29.596320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:29.597259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:29.598018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:29.600602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 98/203] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:29.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:29.728766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:29.756082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba39c66-7f07-4850-bebd-440e6f1b6a42 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:29.789593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:29.790576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:29.790890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:29.791208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:29.791887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:29.792187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:29.793572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:29.808619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 106/204] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:29.814300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7f73a-a55c-4457-a721-3be455ea66fe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:29.956529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7f73a-a55c-4457-a721-3be455ea66fe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:30.001726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858aa8d8-c27b-46ba-96e8-408c2fe500b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:30.002365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aa8d8-c27b-46ba-96e8-408c2fe500b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:30.002640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aa8d8-c27b-46ba-96e8-408c2fe500b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:30.002930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aa8d8-c27b-46ba-96e8-408c2fe500b3 None admin] PATH_INFO: `/v3/users/63ea915cb96141c684220c4df0efa829` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:30.007405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858aa8d8-c27b-46ba-96e8-408c2fe500b3 None admin] RBAC: Authorizing `identity:get_user(user_id=63ea915cb96141c684220c4df0efa829)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:30.008914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858aa8d8-c27b-46ba-96e8-408c2fe500b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:30.010853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 99/205] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:29 2022] GET /identity/v3/users/63ea915cb96141c684220c4df0efa829 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:30.016672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:30.170612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2064e-0e83-43ca-9f14-d4a1922b041d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:30.216968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24da60-dc95-4f79-b5a4-92e0213326a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:30.217510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24da60-dc95-4f79-b5a4-92e0213326a1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:30.217790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24da60-dc95-4f79-b5a4-92e0213326a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:30.218066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24da60-dc95-4f79-b5a4-92e0213326a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:30.226845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:30.297740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24da60-dc95-4f79-b5a4-92e0213326a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:30.299319 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24da60-dc95-4f79-b5a4-92e0213326a1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:30.402977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 107/206] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:30.409213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858aa8d8-c27b-46ba-96e8-408c2fe500b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:30.476955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:30.536521 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858aa8d8-c27b-46ba-96e8-408c2fe500b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:30.581758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30de243-5a3d-4f5d-802f-bd8a8c00ee88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:30.582397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30de243-5a3d-4f5d-802f-bd8a8c00ee88 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:30.582672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30de243-5a3d-4f5d-802f-bd8a8c00ee88 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:30.582965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30de243-5a3d-4f5d-802f-bd8a8c00ee88 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:30.587447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30de243-5a3d-4f5d-802f-bd8a8c00ee88 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:30.589021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30de243-5a3d-4f5d-802f-bd8a8c00ee88 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:30.591063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 100/207] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:30 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:30.597074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24da60-dc95-4f79-b5a4-92e0213326a1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:30.610176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7026e4-ebf7-4557-abcb-70eb4391c4eb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:30.610475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7026e4-ebf7-4557-abcb-70eb4391c4eb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:30.610750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7026e4-ebf7-4557-abcb-70eb4391c4eb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:30.612101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 101/208] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:30.619916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75e7a-2069-4a40-bae4-75f016b2dffd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:30.620194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75e7a-2069-4a40-bae4-75f016b2dffd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:30.620465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b75e7a-2069-4a40-bae4-75f016b2dffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:30.650171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e0b75e7a-2069-4a40-bae4-75f016b2dffd None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:30.651478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0b75e7a-2069-4a40-bae4-75f016b2dffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:30.659114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:30.748657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24da60-dc95-4f79-b5a4-92e0213326a1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:30.750447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 102/209] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:30.808164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28772a50-9337-4f0e-8370-a23ea2fe5848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:30.809195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772a50-9337-4f0e-8370-a23ea2fe5848 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:30.809735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772a50-9337-4f0e-8370-a23ea2fe5848 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:30.810210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772a50-9337-4f0e-8370-a23ea2fe5848 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:30.811168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28772a50-9337-4f0e-8370-a23ea2fe5848 None admin] RBAC: Adding query filter params (role.id=28923fdfbb364d71a285386fd05d9fb2, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf, user.id=63ea915cb96141c684220c4df0efa829) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:30.811669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28772a50-9337-4f0e-8370-a23ea2fe5848 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:30.813331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28772a50-9337-4f0e-8370-a23ea2fe5848 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:30.823236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 108/210] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:30 2022] GET /identity/v3/role_assignments?user.id=63ea915cb96141c684220c4df0efa829&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=28923fdfbb364d71a285386fd05d9fb2 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:30.847047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5a52f-16c2-4ad8-b9fa-994d5617652c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:30.847380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5a52f-16c2-4ad8-b9fa-994d5617652c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:30.847662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5a52f-16c2-4ad8-b9fa-994d5617652c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:30.849052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 103/211] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:30.864559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eb151-794f-4b7f-acea-228bbe58d8f4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:30.865175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eb151-794f-4b7f-acea-228bbe58d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:30.865639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eb151-794f-4b7f-acea-228bbe58d8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:30.891642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-160eb151-794f-4b7f-acea-228bbe58d8f4 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:30.893280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-160eb151-794f-4b7f-acea-228bbe58d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:30.989084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 109/212] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:31.062675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5a52f-16c2-4ad8-b9fa-994d5617652c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:31.120011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:31.168238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5a52f-16c2-4ad8-b9fa-994d5617652c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:31.202874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c5ae8-7acd-4918-a092-7835ddc59cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:31.203516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5ae8-7acd-4918-a092-7835ddc59cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.203793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5ae8-7acd-4918-a092-7835ddc59cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.204060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c5ae8-7acd-4918-a092-7835ddc59cf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.214648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c5ae8-7acd-4918-a092-7835ddc59cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-221c5ae8-7acd-4918-a092-7835ddc59cf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:31.219497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:31.220336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:31.221547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 104/213] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:31.226798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160eb151-794f-4b7f-acea-228bbe58d8f4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:31.283098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:31.303597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c5ae8-7acd-4918-a092-7835ddc59cf6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:31.359961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160eb151-794f-4b7f-acea-228bbe58d8f4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:31.410907 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:31.412982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.413406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.413835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.414673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:31.415223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:31.417820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:31.429209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 110/214] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:31.434602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:31.446877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c5ae8-7acd-4918-a092-7835ddc59cf6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:31.460723 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:31.482088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29169a4c-fd1e-43ce-a9ff-96bdbec6e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:31.482744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169a4c-fd1e-43ce-a9ff-96bdbec6e905 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.483025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169a4c-fd1e-43ce-a9ff-96bdbec6e905 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.483294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169a4c-fd1e-43ce-a9ff-96bdbec6e905 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.484448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29169a4c-fd1e-43ce-a9ff-96bdbec6e905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:31.486031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29169a4c-fd1e-43ce-a9ff-96bdbec6e905 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:31.488063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 105/215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:31.493297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29169a4c-fd1e-43ce-a9ff-96bdbec6e905 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:31.543744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:31.552786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca9bb3-70cf-42b5-a475-216ead444af4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:31.590506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eea82-48e6-4f0f-8269-196a0bbfb78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:31.591558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eea82-48e6-4f0f-8269-196a0bbfb78f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.591969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eea82-48e6-4f0f-8269-196a0bbfb78f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.592371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eea82-48e6-4f0f-8269-196a0bbfb78f None admin] PATH_INFO: `/v3/users/59237e718e5c4c758d0b27a457d4bf51` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.607574 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727eea82-48e6-4f0f-8269-196a0bbfb78f None admin] RBAC: Authorizing `identity:get_user(user_id=59237e718e5c4c758d0b27a457d4bf51)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:31.609356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727eea82-48e6-4f0f-8269-196a0bbfb78f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:31.611878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 111/216] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:31 2022] GET /identity/v3/users/59237e718e5c4c758d0b27a457d4bf51 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:31.617371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eea82-48e6-4f0f-8269-196a0bbfb78f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:31.626333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29169a4c-fd1e-43ce-a9ff-96bdbec6e905 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:31.661502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e21624-9538-43e9-abbe-cce30f0bb5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:31.662143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e21624-9538-43e9-abbe-cce30f0bb5cc None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.662496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e21624-9538-43e9-abbe-cce30f0bb5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.662780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e21624-9538-43e9-abbe-cce30f0bb5cc None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.663557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e21624-9538-43e9-abbe-cce30f0bb5cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:31.665165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e21624-9538-43e9-abbe-cce30f0bb5cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:31.691757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 106/217] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:21:31 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:21:31.692926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4af185-8fc3-4250-962a-caa181fa84ad None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.693185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4af185-8fc3-4250-962a-caa181fa84ad None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.693472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4af185-8fc3-4250-962a-caa181fa84ad None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.699347 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 107/218] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:31.702541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66746-b39f-4517-b368-316b6c220bd7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.702873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66746-b39f-4517-b368-316b6c220bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.703154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66746-b39f-4517-b368-316b6c220bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.725028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-74c66746-b39f-4517-b368-316b6c220bd7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:31.726676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-74c66746-b39f-4517-b368-316b6c220bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:31.741416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eea82-48e6-4f0f-8269-196a0bbfb78f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:31.764604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:31.785218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706495a-cd0d-4874-9e31-4ad62222cc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:31.785845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706495a-cd0d-4874-9e31-4ad62222cc60 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.786123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706495a-cd0d-4874-9e31-4ad62222cc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.786391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706495a-cd0d-4874-9e31-4ad62222cc60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.811896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:31.823242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 108/219] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:31.828792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42418f6-8f7f-466c-8a1f-f62fe70931ba None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.829081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42418f6-8f7f-466c-8a1f-f62fe70931ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.829344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42418f6-8f7f-466c-8a1f-f62fe70931ba None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.830744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 109/220] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:31.836217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37338fe1-e7a5-4bcb-965a-765e67be797b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:31.836525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37338fe1-e7a5-4bcb-965a-765e67be797b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:31.836764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37338fe1-e7a5-4bcb-965a-765e67be797b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:31.857388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-37338fe1-e7a5-4bcb-965a-765e67be797b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:31.858570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-37338fe1-e7a5-4bcb-965a-765e67be797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:31.871876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706495a-cd0d-4874-9e31-4ad62222cc60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:31.873358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706495a-cd0d-4874-9e31-4ad62222cc60 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:31.936550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 110/221] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:31.998234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 112/222] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:32.003950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37338fe1-e7a5-4bcb-965a-765e67be797b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.094698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706495a-cd0d-4874-9e31-4ad62222cc60 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.113703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:32.121966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37338fe1-e7a5-4bcb-965a-765e67be797b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:32.163986 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:32.166323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edae7ee-e299-4b72-8ac3-5725c1317b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:32.167143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edae7ee-e299-4b72-8ac3-5725c1317b8c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:32.167429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edae7ee-e299-4b72-8ac3-5725c1317b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:32.167852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edae7ee-e299-4b72-8ac3-5725c1317b8c None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:32.169358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edae7ee-e299-4b72-8ac3-5725c1317b8c None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:32.171296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edae7ee-e299-4b72-8ac3-5725c1317b8c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:32.173722 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 111/223] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:32.179538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edae7ee-e299-4b72-8ac3-5725c1317b8c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.225471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706495a-cd0d-4874-9e31-4ad62222cc60 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:32.268010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83843fa5-160f-4dab-aa99-599aa385fa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:32.268659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83843fa5-160f-4dab-aa99-599aa385fa75 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:32.268931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83843fa5-160f-4dab-aa99-599aa385fa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:32.269794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83843fa5-160f-4dab-aa99-599aa385fa75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:32.284821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83843fa5-160f-4dab-aa99-599aa385fa75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-83843fa5-160f-4dab-aa99-599aa385fa75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:32.292123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:32.292975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:32.293739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 113/224] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:32.295481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83843fa5-160f-4dab-aa99-599aa385fa75 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.304529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edae7ee-e299-4b72-8ac3-5725c1317b8c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:32.347438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:32.348049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:32.348323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:32.348592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:32.349326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] RBAC: Adding query filter params (role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=a33c93942a544564b55765288214ed83, user.id=59237e718e5c4c758d0b27a457d4bf51) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:32.349627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:32.351236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:32.357380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 112/225] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/role_assignments?user.id=59237e718e5c4c758d0b27a457d4bf51&scope.project.id=a33c93942a544564b55765288214ed83&role.id=5bbcda16701c451f9419aefc4671840e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:32.359603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.391550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:32.429591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83843fa5-160f-4dab-aa99-599aa385fa75 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:32.467153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:32.494645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2311e-6253-4649-9918-aebc54610483 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:32.512248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71374c5-ca67-46c8-b428-069d1b94a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:32.512918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71374c5-ca67-46c8-b428-069d1b94a8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:32.513192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71374c5-ca67-46c8-b428-069d1b94a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:32.513458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71374c5-ca67-46c8-b428-069d1b94a8b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:32.537963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71374c5-ca67-46c8-b428-069d1b94a8b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c71374c5-ca67-46c8-b428-069d1b94a8b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:32.544362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:32.545224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:32.546139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 114/226] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:32.561626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71374c5-ca67-46c8-b428-069d1b94a8b3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.576478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:32.577103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:32.577384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:32.577645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:32.578327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:32.578621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:32.580054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:32.589816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 113/227] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:32.596496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.745931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71374c5-ca67-46c8-b428-069d1b94a8b3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:32.757749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:32.773268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf2685-3121-4979-8285-f888805353fa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:32.818636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:32.819304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:32.819579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:32.819844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:32.820525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:32.820819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:32.822340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:32.839198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 115/228] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:32.845386 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.853245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf36381-0f99-4964-8f5e-7a6062a14ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:32.853942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36381-0f99-4964-8f5e-7a6062a14ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:32.854219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36381-0f99-4964-8f5e-7a6062a14ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:32.854494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36381-0f99-4964-8f5e-7a6062a14ba2 None admin] PATH_INFO: `/v3/users/5556de0aed2b4bbb95048e86a39f24fd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:32.857542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf36381-0f99-4964-8f5e-7a6062a14ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=5556de0aed2b4bbb95048e86a39f24fd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:32.859083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf36381-0f99-4964-8f5e-7a6062a14ba2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:32.860686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 114/229] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/users/5556de0aed2b4bbb95048e86a39f24fd => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:32.866950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf36381-0f99-4964-8f5e-7a6062a14ba2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:32.915103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:33.034968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf36381-0f99-4964-8f5e-7a6062a14ba2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:33.078743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbea22c-3b04-4497-b75c-d9c2e655a018 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:33.080236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d5180-8d75-470d-9ee5-28932ef1d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:33.080880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5180-8d75-470d-9ee5-28932ef1d569 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:33.081144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5180-8d75-470d-9ee5-28932ef1d569 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:33.081414 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5180-8d75-470d-9ee5-28932ef1d569 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:33.084505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d5180-8d75-470d-9ee5-28932ef1d569 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:33.086029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d5180-8d75-470d-9ee5-28932ef1d569 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:33.087938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 115/230] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:33.094151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d5180-8d75-470d-9ee5-28932ef1d569 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:33.130197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:33.179583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcc4e1-d9f9-4656-85f2-4de92428e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:33.180249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc4e1-d9f9-4656-85f2-4de92428e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:33.180537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc4e1-d9f9-4656-85f2-4de92428e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:33.180864 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc4e1-d9f9-4656-85f2-4de92428e5b8 None admin] PATH_INFO: `/v3/users/5556de0aed2b4bbb95048e86a39f24fd` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:33.182095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcc4e1-d9f9-4656-85f2-4de92428e5b8 None admin] RBAC: Authorizing `identity:get_user(user_id=5556de0aed2b4bbb95048e86a39f24fd)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:33.183609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcc4e1-d9f9-4656-85f2-4de92428e5b8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:33.185220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 116/231] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:32 2022] GET /identity/v3/users/5556de0aed2b4bbb95048e86a39f24fd => generated 283 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:33.191355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcc4e1-d9f9-4656-85f2-4de92428e5b8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:33.207392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d5180-8d75-470d-9ee5-28932ef1d569 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:33.243673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63db03-589a-42c2-b168-788a159a7371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:33.244664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63db03-589a-42c2-b168-788a159a7371 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:33.244983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63db03-589a-42c2-b168-788a159a7371 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:33.245306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63db03-589a-42c2-b168-788a159a7371 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83/users/5556de0aed2b4bbb95048e86a39f24fd/roles/d06115dbab78488182bcb432bf9bec11` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:33.247477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63db03-589a-42c2-b168-788a159a7371 None admin] RBAC: Authorizing `identity:create_grant(project_id=a33c93942a544564b55765288214ed83, user_id=5556de0aed2b4bbb95048e86a39f24fd, role_id=d06115dbab78488182bcb432bf9bec11)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:33.249436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63db03-589a-42c2-b168-788a159a7371 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:33.265454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 116/232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:33 2022] PUT /identity/v3/projects/a33c93942a544564b55765288214ed83/users/5556de0aed2b4bbb95048e86a39f24fd/roles/d06115dbab78488182bcb432bf9bec11 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:21:33.421425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcc4e1-d9f9-4656-85f2-4de92428e5b8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:33.423149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:33.466051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86b43a-4ac1-4817-abfc-f42277af3bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:33.466691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86b43a-4ac1-4817-abfc-f42277af3bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:33.466993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86b43a-4ac1-4817-abfc-f42277af3bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:33.467267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86b43a-4ac1-4817-abfc-f42277af3bb0 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:33.468403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86b43a-4ac1-4817-abfc-f42277af3bb0 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:33.469879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86b43a-4ac1-4817-abfc-f42277af3bb0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:33.471899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 117/233] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:33 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:33.477493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63db03-589a-42c2-b168-788a159a7371 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:33.572850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:33.595977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63db03-589a-42c2-b168-788a159a7371 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:33.643538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4f18c-1126-4459-a25f-a38e2b5c57c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:33.644572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f18c-1126-4459-a25f-a38e2b5c57c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:33.645022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f18c-1126-4459-a25f-a38e2b5c57c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:33.645461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f18c-1126-4459-a25f-a38e2b5c57c9 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83/users/5556de0aed2b4bbb95048e86a39f24fd/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:33.647572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4f18c-1126-4459-a25f-a38e2b5c57c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a33c93942a544564b55765288214ed83, user_id=5556de0aed2b4bbb95048e86a39f24fd, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:33.649232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4f18c-1126-4459-a25f-a38e2b5c57c9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:33.680557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 117/234] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:33 2022] PUT /identity/v3/projects/a33c93942a544564b55765288214ed83/users/5556de0aed2b4bbb95048e86a39f24fd/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:34.034423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fa212-63d9-4d4a-9746-cdbacca4d16f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:34.035104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fa212-63d9-4d4a-9746-cdbacca4d16f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:34.035583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fa212-63d9-4d4a-9746-cdbacca4d16f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:34.037177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 118/235] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:34.043553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a10a-99b5-4069-833c-e26ade7d16af None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:34.044104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a10a-99b5-4069-833c-e26ade7d16af None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:34.044567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a10a-99b5-4069-833c-e26ade7d16af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:34.067576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fa01a10a-99b5-4069-833c-e26ade7d16af None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:34.069278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa01a10a-99b5-4069-833c-e26ade7d16af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:34.150790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965de807-a8cf-40b7-9a9e-25ae84027d31 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:34.151416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965de807-a8cf-40b7-9a9e-25ae84027d31 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:34.151894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965de807-a8cf-40b7-9a9e-25ae84027d31 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:34.153744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 119/236] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:34.159409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe82f-e9d9-4510-b393-cdc23ef20896 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:34.160136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe82f-e9d9-4510-b393-cdc23ef20896 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:34.160781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebe82f-e9d9-4510-b393-cdc23ef20896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:34.167345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 118/237] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:34.183383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7cebe82f-e9d9-4510-b393-cdc23ef20896 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:34.184817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7cebe82f-e9d9-4510-b393-cdc23ef20896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:34.254620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 120/238] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:34.499760 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01a10a-99b5-4069-833c-e26ade7d16af None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:34.616866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:34.645205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01a10a-99b5-4069-833c-e26ade7d16af None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:34.690212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769e8b9-2ca0-412b-903f-45084a14713f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:34.691241 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e8b9-2ca0-412b-903f-45084a14713f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:34.691517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e8b9-2ca0-412b-903f-45084a14713f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:34.691782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e8b9-2ca0-412b-903f-45084a14713f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:34.705436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769e8b9-2ca0-412b-903f-45084a14713f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d769e8b9-2ca0-412b-903f-45084a14713f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:34.710270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:34.711121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:34.712385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 119/239] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:34.717738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebe82f-e9d9-4510-b393-cdc23ef20896 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:34.781746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9177-7882-474b-8bb2-766df237cf84 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:34.781935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9177-7882-474b-8bb2-766df237cf84 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:34.782245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9177-7882-474b-8bb2-766df237cf84 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:34.784683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 120/240] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:34.789225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba62f34-cd6e-4766-b601-b2e8a7c7c73a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:34.789502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba62f34-cd6e-4766-b601-b2e8a7c7c73a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:34.789769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba62f34-cd6e-4766-b601-b2e8a7c7c73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:34.811880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dba62f34-cd6e-4766-b601-b2e8a7c7c73a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:34.813209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dba62f34-cd6e-4766-b601-b2e8a7c7c73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:34.843273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebe82f-e9d9-4510-b393-cdc23ef20896 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:34.851437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:34.882685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:34.883703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:34.884487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:34.884957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:34.885922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:34.886411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:34.888068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:34.897834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 121/241] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:34.898924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:34.915559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 121/242] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:34.917019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba62f34-cd6e-4766-b601-b2e8a7c7c73a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:34.992182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:35.046773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3212d26-0daf-4039-a7fd-040e2466fc35 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:35.070305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba62f34-cd6e-4766-b601-b2e8a7c7c73a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:35.110110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a74dda-205a-47f6-a501-a1606b614ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:35.110776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a74dda-205a-47f6-a501-a1606b614ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.111052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a74dda-205a-47f6-a501-a1606b614ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.111317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a74dda-205a-47f6-a501-a1606b614ae3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.113666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a74dda-205a-47f6-a501-a1606b614ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:35.115247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a74dda-205a-47f6-a501-a1606b614ae3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:35.118451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 122/243] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:35.125453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a74dda-205a-47f6-a501-a1606b614ae3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:35.140522 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f89e0-8e30-4b1c-a6d9-a2872e93a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:35.141171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f89e0-8e30-4b1c-a6d9-a2872e93a522 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.141446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f89e0-8e30-4b1c-a6d9-a2872e93a522 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.141715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f89e0-8e30-4b1c-a6d9-a2872e93a522 None admin] PATH_INFO: `/v3/users/9fb3df5598954f56ac22e03ed6c59a67` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.170602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f89e0-8e30-4b1c-a6d9-a2872e93a522 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb3df5598954f56ac22e03ed6c59a67)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:35.172097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f89e0-8e30-4b1c-a6d9-a2872e93a522 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:35.173156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:35.173792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 122/244] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:34 2022] GET /identity/v3/users/9fb3df5598954f56ac22e03ed6c59a67 => generated 297 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:35.179835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f89e0-8e30-4b1c-a6d9-a2872e93a522 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:35.295958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a74dda-205a-47f6-a501-a1606b614ae3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:35.324106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f89e0-8e30-4b1c-a6d9-a2872e93a522 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:35.351099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666cfd-a407-49d8-b235-a9591b3a0394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:35.351727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666cfd-a407-49d8-b235-a9591b3a0394 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.352007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666cfd-a407-49d8-b235-a9591b3a0394 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.352278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666cfd-a407-49d8-b235-a9591b3a0394 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.353242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af666cfd-a407-49d8-b235-a9591b3a0394 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:35.355000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af666cfd-a407-49d8-b235-a9591b3a0394 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:35.356013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:35.368146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6759b1-5ee1-4d6a-8229-e610686b7241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:35.369141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6759b1-5ee1-4d6a-8229-e610686b7241 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.369615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6759b1-5ee1-4d6a-8229-e610686b7241 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.370068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6759b1-5ee1-4d6a-8229-e610686b7241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.386150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 123/245] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:21:35 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:21:35.395600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666cfd-a407-49d8-b235-a9591b3a0394 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:35.455563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6759b1-5ee1-4d6a-8229-e610686b7241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:35.457398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6759b1-5ee1-4d6a-8229-e610686b7241 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:35.559137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:35.569823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666cfd-a407-49d8-b235-a9591b3a0394 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:35.576094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 123/246] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:35.577478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6a0b-d478-4a52-a957-606f72ce507a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.577763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6a0b-d478-4a52-a957-606f72ce507a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.578034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6a0b-d478-4a52-a957-606f72ce507a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.582860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 124/247] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:35.583967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c6a0b-d478-4a52-a957-606f72ce507a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:35.611941 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c43d2-3e64-4983-bd17-f2ad3bf204d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:35.613108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c43d2-3e64-4983-bd17-f2ad3bf204d2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.613586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c43d2-3e64-4983-bd17-f2ad3bf204d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.614043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c43d2-3e64-4983-bd17-f2ad3bf204d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.620576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:35.630061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c43d2-3e64-4983-bd17-f2ad3bf204d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-fe1c43d2-3e64-4983-bd17-f2ad3bf204d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:35.636908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:35.637852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:35.638626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:35.641115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 124/248] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:35.647472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04104aa4-a336-4616-b837-209dec73f109 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.647755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04104aa4-a336-4616-b837-209dec73f109 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.648026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04104aa4-a336-4616-b837-209dec73f109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.670816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-04104aa4-a336-4616-b837-209dec73f109 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:35.672016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-04104aa4-a336-4616-b837-209dec73f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:35.730471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c6a0b-d478-4a52-a957-606f72ce507a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:35.764938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 125/249] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:35.767618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04104aa4-a336-4616-b837-209dec73f109 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:35.773929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec0ef2-4807-42fb-886d-99c8a8152be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:35.774833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec0ef2-4807-42fb-886d-99c8a8152be8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.775174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec0ef2-4807-42fb-886d-99c8a8152be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.775503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec0ef2-4807-42fb-886d-99c8a8152be8 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.776801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec0ef2-4807-42fb-886d-99c8a8152be8 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:35.778365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec0ef2-4807-42fb-886d-99c8a8152be8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:35.780274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 125/250] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:35 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:35.786460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec0ef2-4807-42fb-886d-99c8a8152be8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:35.852329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:35.877345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04104aa4-a336-4616-b837-209dec73f109 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:35.887341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:35.898077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec0ef2-4807-42fb-886d-99c8a8152be8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:35.916959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:35.917944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.918414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.918927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.919818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:35.920321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:35.921929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:35.932112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 126/251] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:35.937759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:35.944304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:35.945188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:35.945659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:35.946118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:35.947182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] RBAC: Adding query filter params (role.id=28923fdfbb364d71a285386fd05d9fb2, scope.project.id=a33c93942a544564b55765288214ed83, user.id=9fb3df5598954f56ac22e03ed6c59a67) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:35.947681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:35.949773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:35.957892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 126/252] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:35 2022] GET /identity/v3/role_assignments?user.id=9fb3df5598954f56ac22e03ed6c59a67&scope.project.id=a33c93942a544564b55765288214ed83&role.id=28923fdfbb364d71a285386fd05d9fb2 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:36.066937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca698b-d338-4e2f-9eef-ac3e7ed5b486 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:36.110043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a04d71-b498-471c-b520-a8d35caa6e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:36.110620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a04d71-b498-471c-b520-a8d35caa6e01 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:36.110928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a04d71-b498-471c-b520-a8d35caa6e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:36.111200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a04d71-b498-471c-b520-a8d35caa6e01 None admin] PATH_INFO: `/v3/users/59237e718e5c4c758d0b27a457d4bf51` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:36.112358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a04d71-b498-471c-b520-a8d35caa6e01 None admin] RBAC: Authorizing `identity:get_user(user_id=59237e718e5c4c758d0b27a457d4bf51)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:36.113823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a04d71-b498-471c-b520-a8d35caa6e01 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:36.115746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 127/253] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:35 2022] GET /identity/v3/users/59237e718e5c4c758d0b27a457d4bf51 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:36.117207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:36.121885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a04d71-b498-471c-b520-a8d35caa6e01 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:36.133298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:36.252316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a04d71-b498-471c-b520-a8d35caa6e01 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:36.272324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0ff32-a6bf-40eb-8ae2-6789a59a34ff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:36.294099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0816e-4af0-40a1-b978-359907d82326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:36.296227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0816e-4af0-40a1-b978-359907d82326 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:36.296514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0816e-4af0-40a1-b978-359907d82326 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:36.296787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0816e-4af0-40a1-b978-359907d82326 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:36.298672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0816e-4af0-40a1-b978-359907d82326 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:36.300161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0816e-4af0-40a1-b978-359907d82326 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:36.302277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 128/254] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:36 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:36.303794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402524d-6623-463f-964c-84757472447d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:36.304284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402524d-6623-463f-964c-84757472447d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:36.304764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402524d-6623-463f-964c-84757472447d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:36.306707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 129/255] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:36.307676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402524d-6623-463f-964c-84757472447d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:36.327033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f20a5-a960-4066-80d5-eb563569108b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:36.327682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f20a5-a960-4066-80d5-eb563569108b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:36.327957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f20a5-a960-4066-80d5-eb563569108b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:36.328225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f20a5-a960-4066-80d5-eb563569108b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:36.351690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f20a5-a960-4066-80d5-eb563569108b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7e7f20a5-a960-4066-80d5-eb563569108b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:36.363735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:36.364656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:36.365432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 127/256] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:36.366058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f750-76d8-4516-9ad9-7e9a852beb0e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:36.374773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f750-76d8-4516-9ad9-7e9a852beb0e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:36.375619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f750-76d8-4516-9ad9-7e9a852beb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:36.407531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fbd0f750-76d8-4516-9ad9-7e9a852beb0e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:36.408735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fbd0f750-76d8-4516-9ad9-7e9a852beb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:36.437026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:36.464921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:36.495311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402524d-6623-463f-964c-84757472447d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:36.532071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0b409-8aa7-4a0c-b84a-9e7e7986a4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:36.533051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0b409-8aa7-4a0c-b84a-9e7e7986a4ae None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:36.533502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0b409-8aa7-4a0c-b84a-9e7e7986a4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:36.533944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0b409-8aa7-4a0c-b84a-9e7e7986a4ae None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83/users/59237e718e5c4c758d0b27a457d4bf51/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:36.536448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0b409-8aa7-4a0c-b84a-9e7e7986a4ae None admin] RBAC: Authorizing `identity:create_grant(project_id=a33c93942a544564b55765288214ed83, user_id=59237e718e5c4c758d0b27a457d4bf51, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:36.538165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0b409-8aa7-4a0c-b84a-9e7e7986a4ae None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:36.573843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 130/257] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:36 2022] PUT /identity/v3/projects/a33c93942a544564b55765288214ed83/users/59237e718e5c4c758d0b27a457d4bf51/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:36.575925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0b409-8aa7-4a0c-b84a-9e7e7986a4ae None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:36.585408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 128/258] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:36.739414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0b409-8aa7-4a0c-b84a-9e7e7986a4ae None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:36.779984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:36.781326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:36.781794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:36.782245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:36.783204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:36.783699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:36.785339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:36.786971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:36.794170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 131/259] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:36.800056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f750-76d8-4516-9ad9-7e9a852beb0e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:36.949325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f750-76d8-4516-9ad9-7e9a852beb0e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:36.991345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148f7c9-f8df-496a-832b-524548309af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:36.991905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148f7c9-f8df-496a-832b-524548309af7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:36.992183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148f7c9-f8df-496a-832b-524548309af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:36.992444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148f7c9-f8df-496a-832b-524548309af7 None admin] PATH_INFO: `/v3/users/5556de0aed2b4bbb95048e86a39f24fd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:36.999381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148f7c9-f8df-496a-832b-524548309af7 None admin] RBAC: Authorizing `identity:get_user(user_id=5556de0aed2b4bbb95048e86a39f24fd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:37.000840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148f7c9-f8df-496a-832b-524548309af7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:37.002651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 129/260] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:36 2022] GET /identity/v3/users/5556de0aed2b4bbb95048e86a39f24fd => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:37.008562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:37.094443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148f7c9-f8df-496a-832b-524548309af7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:37.129131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:37.250099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e92134-b6c9-4e18-896e-5308eb730f81 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:37.327436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406fce-ac1d-4c54-9aa2-f203bce429f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:37.328309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406fce-ac1d-4c54-9aa2-f203bce429f3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:37.328725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406fce-ac1d-4c54-9aa2-f203bce429f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:37.329227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406fce-ac1d-4c54-9aa2-f203bce429f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:37.336853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148f7c9-f8df-496a-832b-524548309af7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:37.395868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42714e82-4dc4-4eb0-8ec5-792f4839c0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:37.396876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42714e82-4dc4-4eb0-8ec5-792f4839c0ad None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:37.397369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42714e82-4dc4-4eb0-8ec5-792f4839c0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:37.397824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42714e82-4dc4-4eb0-8ec5-792f4839c0ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:37.415672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42714e82-4dc4-4eb0-8ec5-792f4839c0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:37.416383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-42714e82-4dc4-4eb0-8ec5-792f4839c0ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:37.421225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:37.422086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:37.423329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 130/261] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:37.427310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c406fce-ac1d-4c54-9aa2-f203bce429f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:37.431183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c406fce-ac1d-4c54-9aa2-f203bce429f3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:37.431822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42714e82-4dc4-4eb0-8ec5-792f4839c0ad None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:37.561260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:37.599072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 132/262] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:37.603373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406fce-ac1d-4c54-9aa2-f203bce429f3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:37.631871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42714e82-4dc4-4eb0-8ec5-792f4839c0ad None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:37.669997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:37.671001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:37.671492 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:37.671970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:37.672828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:37.673716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:37.675410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:37.685614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 131/263] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:37.695806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:37.831810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:37.848805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406fce-ac1d-4c54-9aa2-f203bce429f3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:37.901027 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f5b4-c0ba-471a-82f6-84937607775a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:37.926814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed4016-9bc9-4da9-9c0f-b0ceda2b7cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:37.927420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed4016-9bc9-4da9-9c0f-b0ceda2b7cbe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:37.927700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed4016-9bc9-4da9-9c0f-b0ceda2b7cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:37.927964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed4016-9bc9-4da9-9c0f-b0ceda2b7cbe None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:37.929087 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed4016-9bc9-4da9-9c0f-b0ceda2b7cbe None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:37.930544 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed4016-9bc9-4da9-9c0f-b0ceda2b7cbe None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:37.941111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 133/264] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:37 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:37.953182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed4016-9bc9-4da9-9c0f-b0ceda2b7cbe None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:37.984332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545db160-c825-4f39-a1a0-d36729317fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:37.984981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db160-c825-4f39-a1a0-d36729317fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:37.985260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db160-c825-4f39-a1a0-d36729317fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:37.985528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db160-c825-4f39-a1a0-d36729317fa9 None admin] PATH_INFO: `/v3/users/5556de0aed2b4bbb95048e86a39f24fd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:37.991476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545db160-c825-4f39-a1a0-d36729317fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=5556de0aed2b4bbb95048e86a39f24fd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:37.992992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545db160-c825-4f39-a1a0-d36729317fa9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:37.994595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 132/265] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:37 2022] GET /identity/v3/users/5556de0aed2b4bbb95048e86a39f24fd => generated 283 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:38.000109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545db160-c825-4f39-a1a0-d36729317fa9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:38.006128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:38.150760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed4016-9bc9-4da9-9c0f-b0ceda2b7cbe None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:38.218689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7714dcc-a10f-49ca-87d2-36e0c11edeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:38.219565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7714dcc-a10f-49ca-87d2-36e0c11edeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.221471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7714dcc-a10f-49ca-87d2-36e0c11edeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.221877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7714dcc-a10f-49ca-87d2-36e0c11edeb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.222764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7714dcc-a10f-49ca-87d2-36e0c11edeb5 None admin] RBAC: Adding query filter params (role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=a33c93942a544564b55765288214ed83, user.id=5556de0aed2b4bbb95048e86a39f24fd) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:38.223398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7714dcc-a10f-49ca-87d2-36e0c11edeb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:38.226029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7714dcc-a10f-49ca-87d2-36e0c11edeb5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:38.238512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 134/266] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:37 2022] GET /identity/v3/role_assignments?user.id=5556de0aed2b4bbb95048e86a39f24fd&scope.project.id=a33c93942a544564b55765288214ed83&role.id=d06115dbab78488182bcb432bf9bec11 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:38.255141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545db160-c825-4f39-a1a0-d36729317fa9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:38.294578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d44d40-2f5e-4b43-9ed8-aa84ec6e9dfd None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.295258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d44d40-2f5e-4b43-9ed8-aa84ec6e9dfd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.295747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d44d40-2f5e-4b43-9ed8-aa84ec6e9dfd None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.297440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 135/267] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:38.304433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f7b90-9ed6-4eab-b195-f8ca8c7f76cb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.305113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f7b90-9ed6-4eab-b195-f8ca8c7f76cb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.305903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f7b90-9ed6-4eab-b195-f8ca8c7f76cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.315524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b216f0-be69-439b-977f-9fab86247ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:38.316514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b216f0-be69-439b-977f-9fab86247ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.316788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b216f0-be69-439b-977f-9fab86247ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.317054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b216f0-be69-439b-977f-9fab86247ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.330977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7f2f7b90-9ed6-4eab-b195-f8ca8c7f76cb None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:38.333512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7f2f7b90-9ed6-4eab-b195-f8ca8c7f76cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:38.367090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:38.418624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 136/268] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:38.460170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7387c8-d4d6-4c2d-b1e0-aecddecb19b7 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.461272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7387c8-d4d6-4c2d-b1e0-aecddecb19b7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.461816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7387c8-d4d6-4c2d-b1e0-aecddecb19b7 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.463018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b216f0-be69-439b-977f-9fab86247ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:38.464058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 137/269] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:38.465109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b216f0-be69-439b-977f-9fab86247ae5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:38.469652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e89e-47b9-404b-bcec-b40a04f494d7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.470289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e89e-47b9-404b-bcec-b40a04f494d7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.470810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e89e-47b9-404b-bcec-b40a04f494d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.471896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:38.504322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e251e89e-47b9-404b-bcec-b40a04f494d7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:38.505536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e251e89e-47b9-404b-bcec-b40a04f494d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:38.594287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 138/270] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:38.595723 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504010c2-6721-4e52-b6ca-1aca5ed12300 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.596008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504010c2-6721-4e52-b6ca-1aca5ed12300 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.596386 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504010c2-6721-4e52-b6ca-1aca5ed12300 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.597875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 139/271] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:38.603059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2143e-7e39-46aa-9dc2-56e277721be6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.603357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2143e-7e39-46aa-9dc2-56e277721be6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.603623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2143e-7e39-46aa-9dc2-56e277721be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.608753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 133/272] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 04:21:38.614184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b216f0-be69-439b-977f-9fab86247ae5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:38.644221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-92f2143e-7e39-46aa-9dc2-56e277721be6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:38.645411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-92f2143e-7e39-46aa-9dc2-56e277721be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:38.763421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 140/273] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 31 04:21:38.774745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b216f0-be69-439b-977f-9fab86247ae5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:38.785192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:38.787703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2143e-7e39-46aa-9dc2-56e277721be6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:38.814491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:38.840594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83659358-fa62-4522-8431-8c2b62ff8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:38.841234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83659358-fa62-4522-8431-8c2b62ff8e7b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:38.841506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83659358-fa62-4522-8431-8c2b62ff8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:38.841770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83659358-fa62-4522-8431-8c2b62ff8e7b None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:38.844007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83659358-fa62-4522-8431-8c2b62ff8e7b None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:38.845465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83659358-fa62-4522-8431-8c2b62ff8e7b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:38.847127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 134/274] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:38 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:38.855029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83659358-fa62-4522-8431-8c2b62ff8e7b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:38.966845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83659358-fa62-4522-8431-8c2b62ff8e7b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:38.974538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2143e-7e39-46aa-9dc2-56e277721be6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.004514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.005509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.006001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.006458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.007415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=a33c93942a544564b55765288214ed83, user.id=5556de0aed2b4bbb95048e86a39f24fd) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:39.007910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:39.009689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:39.016301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 135/275] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:38 2022] GET /identity/v3/role_assignments?user.id=5556de0aed2b4bbb95048e86a39f24fd&scope.project.id=a33c93942a544564b55765288214ed83&role.id=241e99d365564c0fa95cb712f4004785 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:39.017378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.018097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26911c-fdbb-4f19-ba3f-54e52d736ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.018756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26911c-fdbb-4f19-ba3f-54e52d736ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.019078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26911c-fdbb-4f19-ba3f-54e52d736ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.019303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26911c-fdbb-4f19-ba3f-54e52d736ab6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.020430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26911c-fdbb-4f19-ba3f-54e52d736ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:39.022901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26911c-fdbb-4f19-ba3f-54e52d736ab6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:39.028056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 141/276] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:39.036494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26911c-fdbb-4f19-ba3f-54e52d736ab6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.105358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:39.142631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:39.158936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26911c-fdbb-4f19-ba3f-54e52d736ab6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.185284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5419792-5148-498f-8b5f-c1eb4a88bbb0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.197034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367bb54-67e2-497b-9e6c-04eb6f959989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.197783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367bb54-67e2-497b-9e6c-04eb6f959989 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.198058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367bb54-67e2-497b-9e6c-04eb6f959989 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.198327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367bb54-67e2-497b-9e6c-04eb6f959989 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.199018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367bb54-67e2-497b-9e6c-04eb6f959989 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:39.200676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367bb54-67e2-497b-9e6c-04eb6f959989 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:39.224872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55d383-eaa0-42e4-836b-c410064f5cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.225519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55d383-eaa0-42e4-836b-c410064f5cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.225793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55d383-eaa0-42e4-836b-c410064f5cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.226061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55d383-eaa0-42e4-836b-c410064f5cc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.231741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 142/277] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:21:39 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:21:39.232171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367bb54-67e2-497b-9e6c-04eb6f959989 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.241996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55d383-eaa0-42e4-836b-c410064f5cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5c55d383-eaa0-42e4-836b-c410064f5cc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:39.247947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:39.248915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:39.249686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:39.250264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 136/278] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:39.260213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55d383-eaa0-42e4-836b-c410064f5cc3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.365506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367bb54-67e2-497b-9e6c-04eb6f959989 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.396633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:39.405161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55d383-eaa0-42e4-836b-c410064f5cc3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.405596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c36f68-919a-42c0-b393-53b9b059c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.406260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c36f68-919a-42c0-b393-53b9b059c894 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.406539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c36f68-919a-42c0-b393-53b9b059c894 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.406862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c36f68-919a-42c0-b393-53b9b059c894 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.422645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c36f68-919a-42c0-b393-53b9b059c894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-31c36f68-919a-42c0-b393-53b9b059c894 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:39.428800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:39.429880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:39.430911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 143/279] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:39.435839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c36f68-919a-42c0-b393-53b9b059c894 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.439572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:39.451560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.452649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.452938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.453207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.453886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:39.454182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:39.456290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:39.467872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 137/280] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:39.474062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.560741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c36f68-919a-42c0-b393-53b9b059c894 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.594706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50273e-3b8b-4b81-9263-7ade12529b60 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.602562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576b335-81c2-4f9b-b158-72dd9d11b54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.603706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576b335-81c2-4f9b-b158-72dd9d11b54c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.603985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576b335-81c2-4f9b-b158-72dd9d11b54c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.604257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576b335-81c2-4f9b-b158-72dd9d11b54c None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.604937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576b335-81c2-4f9b-b158-72dd9d11b54c None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:39.605615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576b335-81c2-4f9b-b158-72dd9d11b54c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:39.607369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576b335-81c2-4f9b-b158-72dd9d11b54c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:39.616361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 144/281] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:39.617400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a3592-ae41-4235-8b95-46e68a0af16b None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.617688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a3592-ae41-4235-8b95-46e68a0af16b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.618077 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a3592-ae41-4235-8b95-46e68a0af16b None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.619487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 145/282] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:39.625402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aaa0a-8f5f-4041-bda5-d57dab1da9aa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.625677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aaa0a-8f5f-4041-bda5-d57dab1da9aa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.625944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aaa0a-8f5f-4041-bda5-d57dab1da9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.636360 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2415e37-7e20-468e-9d19-52164212d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.637005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415e37-7e20-468e-9d19-52164212d5bd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.637282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415e37-7e20-468e-9d19-52164212d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.637558 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415e37-7e20-468e-9d19-52164212d5bd None admin] PATH_INFO: `/v3/users/9fb3df5598954f56ac22e03ed6c59a67` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.638749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2415e37-7e20-468e-9d19-52164212d5bd None admin] RBAC: Authorizing `identity:get_user(user_id=9fb3df5598954f56ac22e03ed6c59a67)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:39.640291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2415e37-7e20-468e-9d19-52164212d5bd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:39.641896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 138/283] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:39 2022] GET /identity/v3/users/9fb3df5598954f56ac22e03ed6c59a67 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:39.642662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2415e37-7e20-468e-9d19-52164212d5bd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.645848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-990aaa0a-8f5f-4041-bda5-d57dab1da9aa None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:39.647783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-990aaa0a-8f5f-4041-bda5-d57dab1da9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:39.682737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:39.738197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 146/284] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:39.739732 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990aaa0a-8f5f-4041-bda5-d57dab1da9aa None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.743157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:39.760215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2415e37-7e20-468e-9d19-52164212d5bd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.819625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b1570-1378-4518-bc4b-dddb5ee72ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.820262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b1570-1378-4518-bc4b-dddb5ee72ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.820535 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b1570-1378-4518-bc4b-dddb5ee72ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.820802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b1570-1378-4518-bc4b-dddb5ee72ad5 None admin] PATH_INFO: `/v3/users/923f5a1abb854e2f8c71ee7e08afa73d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.847641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b1570-1378-4518-bc4b-dddb5ee72ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=923f5a1abb854e2f8c71ee7e08afa73d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:39.850176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b1570-1378-4518-bc4b-dddb5ee72ad5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:39.852072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 139/285] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:39 2022] GET /identity/v3/users/923f5a1abb854e2f8c71ee7e08afa73d => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:39.858455 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b1570-1378-4518-bc4b-dddb5ee72ad5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.903981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990aaa0a-8f5f-4041-bda5-d57dab1da9aa None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:39.955490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:39.965402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893f40-f1a2-481b-a047-bc259e9d24aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:39.966096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893f40-f1a2-481b-a047-bc259e9d24aa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:39.966383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893f40-f1a2-481b-a047-bc259e9d24aa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:39.966658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5893f40-f1a2-481b-a047-bc259e9d24aa None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:39.968453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5893f40-f1a2-481b-a047-bc259e9d24aa None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:39.970044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5893f40-f1a2-481b-a047-bc259e9d24aa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:39.972188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 147/286] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:39 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:39.977787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893f40-f1a2-481b-a047-bc259e9d24aa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:39.986659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b1570-1378-4518-bc4b-dddb5ee72ad5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:40.024576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:40.025257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:40.025531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:40.025807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:40.026641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] RBAC: Adding query filter params (role.id=5bbcda16701c451f9419aefc4671840e, user.id=923f5a1abb854e2f8c71ee7e08afa73d, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:40.026968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:40.028423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:40.033923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 140/287] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Aug 31 04:21:39 2022] GET /identity/v3/role_assignments?user.id=923f5a1abb854e2f8c71ee7e08afa73d&scope.system=all&role.id=5bbcda16701c451f9419aefc4671840e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:40.084284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:40.132601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:40.212212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5893f40-f1a2-481b-a047-bc259e9d24aa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:40.254959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75631c26-31a4-4edb-9800-4945b7e9932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:40.254959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75631c26-31a4-4edb-9800-4945b7e9932f None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:40.255263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75631c26-31a4-4edb-9800-4945b7e9932f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:40.255472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75631c26-31a4-4edb-9800-4945b7e9932f None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83/users/9fb3df5598954f56ac22e03ed6c59a67/roles/28923fdfbb364d71a285386fd05d9fb2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:40.263187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75631c26-31a4-4edb-9800-4945b7e9932f None admin] RBAC: Authorizing `identity:create_grant(project_id=a33c93942a544564b55765288214ed83, user_id=9fb3df5598954f56ac22e03ed6c59a67, role_id=28923fdfbb364d71a285386fd05d9fb2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:40.264936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75631c26-31a4-4edb-9800-4945b7e9932f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:40.266165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa9066-9c1e-4c5b-a7bd-7243a1188dd6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:40.286765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 148/288] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:21:39 2022] PUT /identity/v3/projects/a33c93942a544564b55765288214ed83/users/9fb3df5598954f56ac22e03ed6c59a67/roles/28923fdfbb364d71a285386fd05d9fb2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:21:40.314919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a876c9-20d3-4670-a9b5-3e02bbbe4d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:40.316192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a876c9-20d3-4670-a9b5-3e02bbbe4d84 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:40.316636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a876c9-20d3-4670-a9b5-3e02bbbe4d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:40.317047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a876c9-20d3-4670-a9b5-3e02bbbe4d84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:40.330662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a876c9-20d3-4670-a9b5-3e02bbbe4d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-09a876c9-20d3-4670-a9b5-3e02bbbe4d84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:40.336343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:40.337560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:40.338768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:40.341313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 141/289] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:40.347411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75631c26-31a4-4edb-9800-4945b7e9932f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:40.499789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75631c26-31a4-4edb-9800-4945b7e9932f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:40.528651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:40.541529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:40.542164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:40.542647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:40.543109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:40.544033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:40.544333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:40.545723 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:40.557971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 149/290] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:40.561320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a876c9-20d3-4670-a9b5-3e02bbbe4d84 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:40.580286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:40.702586 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a876c9-20d3-4670-a9b5-3e02bbbe4d84 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:40.753979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be42d40-f264-4169-ba9a-334bd8c4b061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:40.754534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42d40-f264-4169-ba9a-334bd8c4b061 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:40.754833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42d40-f264-4169-ba9a-334bd8c4b061 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:40.755102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42d40-f264-4169-ba9a-334bd8c4b061 None admin] PATH_INFO: `/v3/users/59237e718e5c4c758d0b27a457d4bf51` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:40.757689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be42d40-f264-4169-ba9a-334bd8c4b061 None admin] RBAC: Authorizing `identity:get_user(user_id=59237e718e5c4c758d0b27a457d4bf51)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:40.759297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be42d40-f264-4169-ba9a-334bd8c4b061 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:40.761251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 142/291] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:40 2022] GET /identity/v3/users/59237e718e5c4c758d0b27a457d4bf51 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:40.766866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:40.912072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf216a24-9eac-4b1f-bc7d-3cadfdbf80f9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:40.957057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b34dbb-baae-47f8-913e-8aed31158435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:40.957686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b34dbb-baae-47f8-913e-8aed31158435 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:40.957970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b34dbb-baae-47f8-913e-8aed31158435 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:40.958238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b34dbb-baae-47f8-913e-8aed31158435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:41.017882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:41.038777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b34dbb-baae-47f8-913e-8aed31158435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:41.040249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b34dbb-baae-47f8-913e-8aed31158435 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:41.163560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 150/292] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:41.170220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be42d40-f264-4169-ba9a-334bd8c4b061 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:41.294795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:41.327940 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be42d40-f264-4169-ba9a-334bd8c4b061 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:41.371987 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e643735-178c-410a-ab45-5d71a1cc9d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:41.372629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e643735-178c-410a-ab45-5d71a1cc9d71 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:41.372906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e643735-178c-410a-ab45-5d71a1cc9d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:41.373175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e643735-178c-410a-ab45-5d71a1cc9d71 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:41.379413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e643735-178c-410a-ab45-5d71a1cc9d71 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:41.380890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e643735-178c-410a-ab45-5d71a1cc9d71 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:41.382931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 143/293] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:41 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:41.389133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b34dbb-baae-47f8-913e-8aed31158435 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:41.516930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:41.535193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b34dbb-baae-47f8-913e-8aed31158435 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:41.584573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bb9d2-0781-4944-81a6-666c0f5c760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:41.585191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bb9d2-0781-4944-81a6-666c0f5c760f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:41.585489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bb9d2-0781-4944-81a6-666c0f5c760f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:41.585738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bb9d2-0781-4944-81a6-666c0f5c760f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:41.586475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bb9d2-0781-4944-81a6-666c0f5c760f None admin] RBAC: Adding query filter params (role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=a33c93942a544564b55765288214ed83, user.id=59237e718e5c4c758d0b27a457d4bf51) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:41.586800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bb9d2-0781-4944-81a6-666c0f5c760f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:41.588347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bb9d2-0781-4944-81a6-666c0f5c760f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:41.595182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 151/294] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:41 2022] GET /identity/v3/role_assignments?user.id=59237e718e5c4c758d0b27a457d4bf51&scope.project.id=a33c93942a544564b55765288214ed83&role.id=5bbcda16701c451f9419aefc4671840e => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:41.848394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11de4-511d-468a-ada1-485d68546cca None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:41.848740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11de4-511d-468a-ada1-485d68546cca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:41.848856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11de4-511d-468a-ada1-485d68546cca None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:41.850485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 144/295] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:41.862033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71085aa-9918-4283-ad54-fd658e7c9b56 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:41.862333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71085aa-9918-4283-ad54-fd658e7c9b56 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:41.862582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71085aa-9918-4283-ad54-fd658e7c9b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:41.882233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e71085aa-9918-4283-ad54-fd658e7c9b56 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:41.883440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e71085aa-9918-4283-ad54-fd658e7c9b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:41.887085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f60fb-fb81-472c-95cf-9aedb5a5515d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:41.887368 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f60fb-fb81-472c-95cf-9aedb5a5515d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:41.887642 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f60fb-fb81-472c-95cf-9aedb5a5515d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:41.889209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 145/296] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:41.894370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776f3d-3e9b-4321-93c3-40e64a998c34 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:41.894651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776f3d-3e9b-4321-93c3-40e64a998c34 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:41.894961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776f3d-3e9b-4321-93c3-40e64a998c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:41.914143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f5776f3d-3e9b-4321-93c3-40e64a998c34 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:41.915591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f5776f3d-3e9b-4321-93c3-40e64a998c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:41.967738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 152/297] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:41.993485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 146/298] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:42.245527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71085aa-9918-4283-ad54-fd658e7c9b56 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:42.282706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5776f3d-3e9b-4321-93c3-40e64a998c34 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:42.339727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:42.373435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71085aa-9918-4283-ad54-fd658e7c9b56 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:42.376481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:42.406148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5776f3d-3e9b-4321-93c3-40e64a998c34 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:42.421227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a7ed-2667-4149-b794-ba60dfb99aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:42.423697 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a7ed-2667-4149-b794-ba60dfb99aaf None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.424416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a7ed-2667-4149-b794-ba60dfb99aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.424713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a7ed-2667-4149-b794-ba60dfb99aaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.439327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637a7ed-2667-4149-b794-ba60dfb99aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-1637a7ed-2667-4149-b794-ba60dfb99aaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:42.445379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:42.446228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:42.447103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:42.449297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cf1c-8ec5-4f93-b57d-3c80e0f01b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:42.450248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cf1c-8ec5-4f93-b57d-3c80e0f01b22 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.450827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cf1c-8ec5-4f93-b57d-3c80e0f01b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.451286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cf1c-8ec5-4f93-b57d-3c80e0f01b22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.451567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 153/299] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:42.452798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3cf1c-8ec5-4f93-b57d-3c80e0f01b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:42.452916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a7ed-2667-4149-b794-ba60dfb99aaf None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:42.454690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3cf1c-8ec5-4f93-b57d-3c80e0f01b22 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:42.456881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 147/300] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:42.462490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cf1c-8ec5-4f93-b57d-3c80e0f01b22 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:42.563792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a7ed-2667-4149-b794-ba60dfb99aaf None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:42.569297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cf1c-8ec5-4f93-b57d-3c80e0f01b22 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:42.601506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ef825-1a0c-455e-83f0-2b5741028733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:42.602128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef825-1a0c-455e-83f0-2b5741028733 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.602413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef825-1a0c-455e-83f0-2b5741028733 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.602675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ef825-1a0c-455e-83f0-2b5741028733 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.603460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ef825-1a0c-455e-83f0-2b5741028733 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:42.603937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ef825-1a0c-455e-83f0-2b5741028733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:42.605326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ef825-1a0c-455e-83f0-2b5741028733 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:42.606842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f700d8-00f3-4447-be4f-990dd429e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:42.607454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f700d8-00f3-4447-be4f-990dd429e72d None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.607725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f700d8-00f3-4447-be4f-990dd429e72d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.607988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f700d8-00f3-4447-be4f-990dd429e72d None admin] PATH_INFO: `/v3/groups` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.608614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f700d8-00f3-4447-be4f-990dd429e72d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:42.610013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f700d8-00f3-4447-be4f-990dd429e72d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:42.610444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:42.613981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 154/301] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:42.615358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aab884-d532-42ff-a805-e23d062d1644 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.615999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aab884-d532-42ff-a805-e23d062d1644 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.616601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aab884-d532-42ff-a805-e23d062d1644 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.618359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 155/302] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:42.619402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fa9d-dd47-45ca-ae5f-ece992141737 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.619699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fa9d-dd47-45ca-ae5f-ece992141737 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.619970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fa9d-dd47-45ca-ae5f-ece992141737 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.621167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 156/303] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:42.622035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923fa9d-dd47-45ca-ae5f-ece992141737 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:42.635789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 148/304] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Aug 31 04:21:42 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:21:42.637343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59e4d9-03d6-421d-9654-7198ec268663 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.637628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59e4d9-03d6-421d-9654-7198ec268663 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.637901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59e4d9-03d6-421d-9654-7198ec268663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.644143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:42.659306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-af59e4d9-03d6-421d-9654-7198ec268663 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:42.660541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-af59e4d9-03d6-421d-9654-7198ec268663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:42.737731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 149/305] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:42.738842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43f4a2-96b1-4c7c-872f-9140b254a27e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.739125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43f4a2-96b1-4c7c-872f-9140b254a27e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.739391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43f4a2-96b1-4c7c-872f-9140b254a27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.760923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5e43f4a2-96b1-4c7c-872f-9140b254a27e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:42.762149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e43f4a2-96b1-4c7c-872f-9140b254a27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:42.770015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923fa9d-dd47-45ca-ae5f-ece992141737 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:42.805490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674e68-dd24-4dc3-819d-d81fc048dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:42.806209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674e68-dd24-4dc3-819d-d81fc048dce9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.806506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674e68-dd24-4dc3-819d-d81fc048dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.806759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674e68-dd24-4dc3-819d-d81fc048dce9 None admin] PATH_INFO: `/v3/users/747e5f18d70e443193439fdef9d0c6b3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.821377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674e68-dd24-4dc3-819d-d81fc048dce9 None admin] RBAC: Authorizing `identity:get_user(user_id=747e5f18d70e443193439fdef9d0c6b3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:42.823176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674e68-dd24-4dc3-819d-d81fc048dce9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:42.825140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 157/306] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:42 2022] GET /identity/v3/users/747e5f18d70e443193439fdef9d0c6b3 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:42.831474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674e68-dd24-4dc3-819d-d81fc048dce9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:42.846300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 150/307] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:42.892959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:42.934673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674e68-dd24-4dc3-819d-d81fc048dce9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:42.968708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:42.969319 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:42.969592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:42.969856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:42.970577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] RBAC: Adding query filter params (role.id=28923fdfbb364d71a285386fd05d9fb2, user.id=747e5f18d70e443193439fdef9d0c6b3, scope.system=all) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:42.970902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:42.972340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:42.977673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 158/308] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Aug 31 04:21:42 2022] GET /identity/v3/role_assignments?user.id=747e5f18d70e443193439fdef9d0c6b3&scope.system=all&role.id=28923fdfbb364d71a285386fd05d9fb2 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:43.008441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43f4a2-96b1-4c7c-872f-9140b254a27e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:43.082570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:43.117047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:43.136028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43f4a2-96b1-4c7c-872f-9140b254a27e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:43.173168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d74134-fbdd-4bf1-88f7-5889908b5ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:43.174513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d74134-fbdd-4bf1-88f7-5889908b5ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:43.174513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d74134-fbdd-4bf1-88f7-5889908b5ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:43.174513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d74134-fbdd-4bf1-88f7-5889908b5ba4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:43.185601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d74134-fbdd-4bf1-88f7-5889908b5ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-84d74134-fbdd-4bf1-88f7-5889908b5ba4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:43.190442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:43.191866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:43.192868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 151/309] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:43.198247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d74134-fbdd-4bf1-88f7-5889908b5ba4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:43.259791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21043d7-ca26-4c90-a083-ef599f48f9a2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:43.291937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:43.312551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0929e-f5b0-49c4-a90f-6b0a736724d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:43.313735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0929e-f5b0-49c4-a90f-6b0a736724d5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:43.314209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0929e-f5b0-49c4-a90f-6b0a736724d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:43.314668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0929e-f5b0-49c4-a90f-6b0a736724d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:43.314965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d74134-fbdd-4bf1-88f7-5889908b5ba4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:43.330381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0929e-f5b0-49c4-a90f-6b0a736724d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d2e0929e-f5b0-49c4-a90f-6b0a736724d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:43.337485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:43.338329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:43.339149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:43.343831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:43.348111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 159/310] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:43.350170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0929e-f5b0-49c4-a90f-6b0a736724d5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:43.366132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:43.367507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:43.368253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:43.368831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:43.370337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:43.371289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:43.373411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:43.386251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 152/311] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:43.391853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:43.477311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0929e-f5b0-49c4-a90f-6b0a736724d5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:43.505282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c4607-f087-4517-a828-177dabdbe188 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:43.523507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:43.524420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:43.524982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:43.525447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:43.526964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:43.527684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:43.530038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:43.540659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620962fc-097b-46f5-8ec3-ce048feae764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:43.541473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620962fc-097b-46f5-8ec3-ce048feae764 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:43.546195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620962fc-097b-46f5-8ec3-ce048feae764 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:43.546195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620962fc-097b-46f5-8ec3-ce048feae764 None admin] PATH_INFO: `/v3/users/923f5a1abb854e2f8c71ee7e08afa73d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:43.546361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 160/312] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:43.546855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620962fc-097b-46f5-8ec3-ce048feae764 None admin] RBAC: Authorizing `identity:get_user(user_id=923f5a1abb854e2f8c71ee7e08afa73d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:43.548334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620962fc-097b-46f5-8ec3-ce048feae764 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:43.550043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 153/313] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:43 2022] GET /identity/v3/users/923f5a1abb854e2f8c71ee7e08afa73d => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:43.551772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:43.557974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620962fc-097b-46f5-8ec3-ce048feae764 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:43.599200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:43.627312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:43.654647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2b1e0-7342-4bc7-bbd7-71342e64b31f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:43.687802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855472d-555f-458e-9fc3-45627750d95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:43.688440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855472d-555f-458e-9fc3-45627750d95c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:43.688718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855472d-555f-458e-9fc3-45627750d95c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:43.689047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855472d-555f-458e-9fc3-45627750d95c None admin] PATH_INFO: `/v3/users/9fb3df5598954f56ac22e03ed6c59a67` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:43.690081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620962fc-097b-46f5-8ec3-ce048feae764 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:43.690253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855472d-555f-458e-9fc3-45627750d95c None admin] RBAC: Authorizing `identity:get_user(user_id=9fb3df5598954f56ac22e03ed6c59a67)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:43.692343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855472d-555f-458e-9fc3-45627750d95c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:43.698634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 161/314] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:43 2022] GET /identity/v3/users/9fb3df5598954f56ac22e03ed6c59a67 => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:43.700377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855472d-555f-458e-9fc3-45627750d95c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:43.732712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439bfb7-7efa-4785-90b8-5546f913e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:43.733366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439bfb7-7efa-4785-90b8-5546f913e416 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:43.733702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439bfb7-7efa-4785-90b8-5546f913e416 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:43.733980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439bfb7-7efa-4785-90b8-5546f913e416 None admin] PATH_INFO: `/v3/system/users/923f5a1abb854e2f8c71ee7e08afa73d/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:43.735448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439bfb7-7efa-4785-90b8-5546f913e416 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=923f5a1abb854e2f8c71ee7e08afa73d, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:43.736841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439bfb7-7efa-4785-90b8-5546f913e416 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:43.752873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 154/315] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Aug 31 04:21:43 2022] PUT /identity/v3/system/users/923f5a1abb854e2f8c71ee7e08afa73d/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:21:43.828116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855472d-555f-458e-9fc3-45627750d95c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:43.845026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:43.870502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd6834-f299-41d6-98f1-83b595d9d87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:43.871511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd6834-f299-41d6-98f1-83b595d9d87d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:43.871959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd6834-f299-41d6-98f1-83b595d9d87d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:43.872401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd6834-f299-41d6-98f1-83b595d9d87d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:43.953866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd6834-f299-41d6-98f1-83b595d9d87d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:43.955535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd6834-f299-41d6-98f1-83b595d9d87d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:44.062615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 162/316] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 04:21:44.069470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439bfb7-7efa-4785-90b8-5546f913e416 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:44.183001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439bfb7-7efa-4785-90b8-5546f913e416 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:44.200227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:44.219342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b769e2-68f4-4387-8120-1974729b885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:44.219980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b769e2-68f4-4387-8120-1974729b885a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:44.220264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b769e2-68f4-4387-8120-1974729b885a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:44.220534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b769e2-68f4-4387-8120-1974729b885a None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:44.221707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b769e2-68f4-4387-8120-1974729b885a None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:44.223196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b769e2-68f4-4387-8120-1974729b885a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:44.230046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 155/317] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:44 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:44.231483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd6834-f299-41d6-98f1-83b595d9d87d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:44.246303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:44.333652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd6834-f299-41d6-98f1-83b595d9d87d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:44.369728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3e63c-e95d-408b-aeee-945c829f2f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:44.370350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3e63c-e95d-408b-aeee-945c829f2f94 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:44.370620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3e63c-e95d-408b-aeee-945c829f2f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:44.370912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3e63c-e95d-408b-aeee-945c829f2f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:44.371652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3e63c-e95d-408b-aeee-945c829f2f94 None admin] RBAC: Adding query filter params (role.id=28923fdfbb364d71a285386fd05d9fb2, scope.project.id=a33c93942a544564b55765288214ed83, user.id=9fb3df5598954f56ac22e03ed6c59a67) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:44.371952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3e63c-e95d-408b-aeee-945c829f2f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:44.373399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3e63c-e95d-408b-aeee-945c829f2f94 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:44.379999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 163/318] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:21:44 2022] GET /identity/v3/role_assignments?user.id=9fb3df5598954f56ac22e03ed6c59a67&scope.project.id=a33c93942a544564b55765288214ed83&role.id=28923fdfbb364d71a285386fd05d9fb2 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:44.695181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883a4b-2c1e-4bc1-8a89-b11d91d2fd73 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:44.695811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883a4b-2c1e-4bc1-8a89-b11d91d2fd73 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:44.695811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883a4b-2c1e-4bc1-8a89-b11d91d2fd73 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:44.697529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 156/319] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:44.702649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a57b1-549c-401b-826f-4221d0f42a8c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:44.702967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a57b1-549c-401b-826f-4221d0f42a8c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:44.703236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a57b1-549c-401b-826f-4221d0f42a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:44.721681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-359a57b1-549c-401b-826f-4221d0f42a8c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:44.722879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-359a57b1-549c-401b-826f-4221d0f42a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:44.790280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 164/320] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:44.946959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0abfb4-66cc-47f9-b43f-87e2324f4be2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:44.947201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0abfb4-66cc-47f9-b43f-87e2324f4be2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:44.947338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0abfb4-66cc-47f9-b43f-87e2324f4be2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:44.948887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 157/321] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:44.953966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d09eef-3108-41e9-b4ef-df8bf3019b0e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:44.954255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d09eef-3108-41e9-b4ef-df8bf3019b0e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:44.954499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d09eef-3108-41e9-b4ef-df8bf3019b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:44.967378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:44.974501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-10d09eef-3108-41e9-b4ef-df8bf3019b0e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:44.975712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-10d09eef-3108-41e9-b4ef-df8bf3019b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:45.043414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 165/322] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:45.054162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0abfb4-66cc-47f9-b43f-87e2324f4be2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:45.155569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0abfb4-66cc-47f9-b43f-87e2324f4be2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:45.189874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989f310-349e-4955-b293-c9216e6e8b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:45.190570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989f310-349e-4955-b293-c9216e6e8b51 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:45.191230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989f310-349e-4955-b293-c9216e6e8b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:45.191502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989f310-349e-4955-b293-c9216e6e8b51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:45.192623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989f310-349e-4955-b293-c9216e6e8b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:45.194067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989f310-349e-4955-b293-c9216e6e8b51 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:45.195826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 158/323] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:21:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:45.202027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d09eef-3108-41e9-b4ef-df8bf3019b0e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:45.310830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d09eef-3108-41e9-b4ef-df8bf3019b0e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:45.320632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989f310-349e-4955-b293-c9216e6e8b51 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:45.350778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9055f-fb21-4b2b-b3ea-b9f6d6292cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:45.351404 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9055f-fb21-4b2b-b3ea-b9f6d6292cab None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:45.351675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9055f-fb21-4b2b-b3ea-b9f6d6292cab None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:45.351942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9055f-fb21-4b2b-b3ea-b9f6d6292cab None admin] PATH_INFO: `/v3/groups` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:45.352566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9055f-fb21-4b2b-b3ea-b9f6d6292cab None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:45.354237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9055f-fb21-4b2b-b3ea-b9f6d6292cab None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:45.360189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:45.374820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 166/324] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Aug 31 04:21:45 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:21:45.383413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:45.440154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989f310-349e-4955-b293-c9216e6e8b51 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:45.477565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a67ab7-f752-4904-99b7-8d8be3259849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:45.478153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a67ab7-f752-4904-99b7-8d8be3259849 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:45.478628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a67ab7-f752-4904-99b7-8d8be3259849 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:45.479041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a67ab7-f752-4904-99b7-8d8be3259849 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:45.492031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a67ab7-f752-4904-99b7-8d8be3259849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a9a67ab7-f752-4904-99b7-8d8be3259849 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:45.497840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:45.499114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:45.499911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:45.503126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 159/325] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:45.505068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9055f-fb21-4b2b-b3ea-b9f6d6292cab None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:45.614300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9055f-fb21-4b2b-b3ea-b9f6d6292cab None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:45.650212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213fe5b-28a2-4464-9604-9287757e9490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:45.650860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213fe5b-28a2-4464-9604-9287757e9490 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:45.651229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213fe5b-28a2-4464-9604-9287757e9490 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:45.651504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213fe5b-28a2-4464-9604-9287757e9490 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:45.652186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213fe5b-28a2-4464-9604-9287757e9490 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:45.652485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213fe5b-28a2-4464-9604-9287757e9490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:45.653868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213fe5b-28a2-4464-9604-9287757e9490 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:45.663062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 167/326] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:45.672283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a67ab7-f752-4904-99b7-8d8be3259849 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:45.775234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31002f-8152-4b0f-9aa5-fbf0b0ae1a7e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:45.775420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31002f-8152-4b0f-9aa5-fbf0b0ae1a7e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:45.775696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31002f-8152-4b0f-9aa5-fbf0b0ae1a7e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:45.777049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 168/327] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:45.782485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414c694-237b-4af0-9842-a5acc999f808 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:45.782780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414c694-237b-4af0-9842-a5acc999f808 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:45.783056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414c694-237b-4af0-9842-a5acc999f808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:45.798261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a67ab7-f752-4904-99b7-8d8be3259849 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:45.807462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e414c694-237b-4af0-9842-a5acc999f808 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:45.808650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e414c694-237b-4af0-9842-a5acc999f808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:45.813815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:45.842670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8701f30-174d-4aea-974f-d64da184738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:45.843517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8701f30-174d-4aea-974f-d64da184738c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:45.843798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8701f30-174d-4aea-974f-d64da184738c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:45.844073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8701f30-174d-4aea-974f-d64da184738c None admin] PATH_INFO: `/v3/users/747e5f18d70e443193439fdef9d0c6b3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:45.845861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8701f30-174d-4aea-974f-d64da184738c None admin] RBAC: Authorizing `identity:get_user(user_id=747e5f18d70e443193439fdef9d0c6b3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:45.847641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8701f30-174d-4aea-974f-d64da184738c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:45.850092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 160/328] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:45 2022] GET /identity/v3/users/747e5f18d70e443193439fdef9d0c6b3 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:45.856279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8701f30-174d-4aea-974f-d64da184738c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:45.907106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:45.909929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 169/329] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:45.973163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8701f30-174d-4aea-974f-d64da184738c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:46.008770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa068dd3-d91c-4f4f-acea-e5ee9a18beb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:46.010378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa068dd3-d91c-4f4f-acea-e5ee9a18beb6 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:46.010836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa068dd3-d91c-4f4f-acea-e5ee9a18beb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:46.011252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa068dd3-d91c-4f4f-acea-e5ee9a18beb6 None admin] PATH_INFO: `/v3/system/users/747e5f18d70e443193439fdef9d0c6b3/roles/28923fdfbb364d71a285386fd05d9fb2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:46.012848 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa068dd3-d91c-4f4f-acea-e5ee9a18beb6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=747e5f18d70e443193439fdef9d0c6b3, role_id=28923fdfbb364d71a285386fd05d9fb2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:46.014373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa068dd3-d91c-4f4f-acea-e5ee9a18beb6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:46.026807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 161/330] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Aug 31 04:21:45 2022] PUT /identity/v3/system/users/747e5f18d70e443193439fdef9d0c6b3/roles/28923fdfbb364d71a285386fd05d9fb2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:21:46.215963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414c694-237b-4af0-9842-a5acc999f808 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:46.341806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414c694-237b-4af0-9842-a5acc999f808 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:46.378860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05efe3f-5e11-40b0-8295-982439d14e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:46.380184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05efe3f-5e11-40b0-8295-982439d14e24 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:46.380601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05efe3f-5e11-40b0-8295-982439d14e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:46.381006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05efe3f-5e11-40b0-8295-982439d14e24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:46.392788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05efe3f-5e11-40b0-8295-982439d14e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b05efe3f-5e11-40b0-8295-982439d14e24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:46.397972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:46.398985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:46.399791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:46.403250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 170/331] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:46.408533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa068dd3-d91c-4f4f-acea-e5ee9a18beb6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:46.482567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:46.562998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa068dd3-d91c-4f4f-acea-e5ee9a18beb6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:46.608263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:46.608808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:46.609081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:46.609348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:46.610025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:46.610323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:46.611844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:46.621050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 162/332] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:46.627470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05efe3f-5e11-40b0-8295-982439d14e24 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:46.697564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:46.773333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05efe3f-5e11-40b0-8295-982439d14e24 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:46.814642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80501e2-5ede-4d71-afa8-9fe0e9107a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:46.815317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80501e2-5ede-4d71-afa8-9fe0e9107a17 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:46.815590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80501e2-5ede-4d71-afa8-9fe0e9107a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:46.815855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80501e2-5ede-4d71-afa8-9fe0e9107a17 None admin] PATH_INFO: `/v3/users/923f5a1abb854e2f8c71ee7e08afa73d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:46.817339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80501e2-5ede-4d71-afa8-9fe0e9107a17 None admin] RBAC: Authorizing `identity:get_user(user_id=923f5a1abb854e2f8c71ee7e08afa73d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:46.818807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80501e2-5ede-4d71-afa8-9fe0e9107a17 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:46.820690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 171/333] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:46 2022] GET /identity/v3/users/923f5a1abb854e2f8c71ee7e08afa73d => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:46.826945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:46.970686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724a93e-25e1-4199-bcf8-403282cfab5f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:46.978785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:47.006428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dc708-1b53-4b25-85b6-5b43e605cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:47.007404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc708-1b53-4b25-85b6-5b43e605cf60 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:47.007870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc708-1b53-4b25-85b6-5b43e605cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:47.008372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc708-1b53-4b25-85b6-5b43e605cf60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:47.009581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66dc708-1b53-4b25-85b6-5b43e605cf60 None admin] RBAC: Adding query filter params (role.id=5bbcda16701c451f9419aefc4671840e, user.id=923f5a1abb854e2f8c71ee7e08afa73d, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:47.010132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66dc708-1b53-4b25-85b6-5b43e605cf60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:47.011798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66dc708-1b53-4b25-85b6-5b43e605cf60 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:47.025353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 163/334] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Aug 31 04:21:46 2022] GET /identity/v3/role_assignments?user.id=923f5a1abb854e2f8c71ee7e08afa73d&scope.system=all&role.id=5bbcda16701c451f9419aefc4671840e => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:47.811031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c967-adb2-4e85-9596-f9ee99642321 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:47.811553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c967-adb2-4e85-9596-f9ee99642321 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:47.811553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c967-adb2-4e85-9596-f9ee99642321 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:47.813182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 172/335] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:47.818622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3456e-3afa-4289-937d-6ca2239a42f5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:47.818935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3456e-3afa-4289-937d-6ca2239a42f5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:47.819205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3456e-3afa-4289-937d-6ca2239a42f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:47.843137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-80d3456e-3afa-4289-937d-6ca2239a42f5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:47.844306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-80d3456e-3afa-4289-937d-6ca2239a42f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:47.916521 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 164/336] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:47.980403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601cb6d-3c12-4fc6-b69f-37a4eecde803 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:47.980711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601cb6d-3c12-4fc6-b69f-37a4eecde803 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:47.980943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601cb6d-3c12-4fc6-b69f-37a4eecde803 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:47.982490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 173/337] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:47.992367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ec8e3-4d23-4400-a956-ccb0a4b6289d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:47.992651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ec8e3-4d23-4400-a956-ccb0a4b6289d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:47.992922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ec8e3-4d23-4400-a956-ccb0a4b6289d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:48.015222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a06ec8e3-4d23-4400-a956-ccb0a4b6289d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:48.016404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a06ec8e3-4d23-4400-a956-ccb0a4b6289d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:48.100603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 165/338] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:48.391835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601cb6d-3c12-4fc6-b69f-37a4eecde803 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:48.513001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601cb6d-3c12-4fc6-b69f-37a4eecde803 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:48.533046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:48.552573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3462b-d9b4-4573-895a-3d3fdb39d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:48.553214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3462b-d9b4-4573-895a-3d3fdb39d661 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:48.553488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3462b-d9b4-4573-895a-3d3fdb39d661 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:48.553753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3462b-d9b4-4573-895a-3d3fdb39d661 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:48.561253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499911fc-ea25-434f-b141-d92ac9d1d858 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:48.561570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499911fc-ea25-434f-b141-d92ac9d1d858 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:48.561804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499911fc-ea25-434f-b141-d92ac9d1d858 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:48.565298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 166/339] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:48.569404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321b222-3d6f-416e-8545-732117a3917f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:48.569689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321b222-3d6f-416e-8545-732117a3917f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:48.569961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321b222-3d6f-416e-8545-732117a3917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:48.575257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3462b-d9b4-4573-895a-3d3fdb39d661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-46f3462b-d9b4-4573-895a-3d3fdb39d661 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:48.580441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:48.581269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:48.582057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:48.584248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 174/340] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:48.595041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3462b-d9b4-4573-895a-3d3fdb39d661 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:48.603363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0321b222-3d6f-416e-8545-732117a3917f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:48.606194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0321b222-3d6f-416e-8545-732117a3917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:48.651052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:48.733885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 167/341] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:48.738939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321b222-3d6f-416e-8545-732117a3917f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:48.773720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3462b-d9b4-4573-895a-3d3fdb39d661 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:48.811377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:48.812032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:48.812261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:48.812511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:48.813187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:48.813480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:48.814896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:48.824155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 175/342] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:48.829562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:48.863843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321b222-3d6f-416e-8545-732117a3917f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:48.907308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:48.952088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423eea8-3ce0-43ef-80d5-682789351227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:48.952745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423eea8-3ce0-43ef-80d5-682789351227 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:48.953022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423eea8-3ce0-43ef-80d5-682789351227 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:48.953291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423eea8-3ce0-43ef-80d5-682789351227 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:48.973166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423eea8-3ce0-43ef-80d5-682789351227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f423eea8-3ce0-43ef-80d5-682789351227 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:48.978745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:48.980010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:48.980774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 168/343] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:48.985861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423eea8-3ce0-43ef-80d5-682789351227 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:49.015284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:49.032707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b158a-6d7e-4d9a-a078-5d16030217a7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:49.118139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bb94c-e438-4471-a55f-23ae5e772be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:49.119354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bb94c-e438-4471-a55f-23ae5e772be8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.119830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bb94c-e438-4471-a55f-23ae5e772be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.120293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bb94c-e438-4471-a55f-23ae5e772be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.160808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423eea8-3ce0-43ef-80d5-682789351227 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:49.205965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:49.206579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.206957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.207232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.210739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:49.210739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:49.210739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:49.219890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 169/344] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:49.225721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:49.250667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bb94c-e438-4471-a55f-23ae5e772be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:49.252409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bb94c-e438-4471-a55f-23ae5e772be8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:49.326906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:49.381450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:49.428559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42462cf-b935-4c8e-a5a3-adcb9bed5ab7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:49.451343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 176/345] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:49.452499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bb94c-e438-4471-a55f-23ae5e772be8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:49.487850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732c71e-cf85-47e7-b3c0-c83a17c7b0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:49.488582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732c71e-cf85-47e7-b3c0-c83a17c7b0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.488748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732c71e-cf85-47e7-b3c0-c83a17c7b0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.489023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732c71e-cf85-47e7-b3c0-c83a17c7b0c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.559851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732c71e-cf85-47e7-b3c0-c83a17c7b0c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:49.561326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732c71e-cf85-47e7-b3c0-c83a17c7b0c1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:49.565058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bb94c-e438-4471-a55f-23ae5e772be8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:49.602746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cae85d-20c2-48cd-8d93-842d216b06a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:49.602746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae85d-20c2-48cd-8d93-842d216b06a6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.602746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae85d-20c2-48cd-8d93-842d216b06a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.602746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae85d-20c2-48cd-8d93-842d216b06a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.611565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:49.615682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cae85d-20c2-48cd-8d93-842d216b06a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d8cae85d-20c2-48cd-8d93-842d216b06a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:49.621684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:49.622908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:49.623691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 177/346] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:49.624320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59d50-f697-4208-ba35-d14886200a75 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.624748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59d50-f697-4208-ba35-d14886200a75 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.625379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59d50-f697-4208-ba35-d14886200a75 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.630860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 178/347] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:49.630860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd59d50-f697-4208-ba35-d14886200a75 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:49.664762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:49.696895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 170/348] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:49.698654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd899af9-fac3-4707-b912-bf49c5e6b0c7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.699174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd899af9-fac3-4707-b912-bf49c5e6b0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.699632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd899af9-fac3-4707-b912-bf49c5e6b0c7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.701347 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 171/349] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:49.702575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff4d92-17fd-480a-ae78-7a35c227b5a1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.706776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff4d92-17fd-480a-ae78-7a35c227b5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.707072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff4d92-17fd-480a-ae78-7a35c227b5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.733527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d4ff4d92-17fd-480a-ae78-7a35c227b5a1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:49.734704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d4ff4d92-17fd-480a-ae78-7a35c227b5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:49.740628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd59d50-f697-4208-ba35-d14886200a75 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:49.787183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3452c-c165-4cf8-a4fb-bb2669cd5d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:49.788438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3452c-c165-4cf8-a4fb-bb2669cd5d12 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.789090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3452c-c165-4cf8-a4fb-bb2669cd5d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.789562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3452c-c165-4cf8-a4fb-bb2669cd5d12 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.791083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3452c-c165-4cf8-a4fb-bb2669cd5d12 None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:49.792966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3452c-c165-4cf8-a4fb-bb2669cd5d12 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:49.795940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 179/350] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:49 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:49.796246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3452c-c165-4cf8-a4fb-bb2669cd5d12 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:49.830126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 172/351] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:49.830884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff4d92-17fd-480a-ae78-7a35c227b5a1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:49.888118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:49.941703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3452c-c165-4cf8-a4fb-bb2669cd5d12 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:49.960083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff4d92-17fd-480a-ae78-7a35c227b5a1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:49.969973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:49.984745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983b1dd-5642-4466-b395-787b5fecc3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:49.986468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983b1dd-5642-4466-b395-787b5fecc3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.986468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983b1dd-5642-4466-b395-787b5fecc3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.986468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983b1dd-5642-4466-b395-787b5fecc3c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:49.986892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983b1dd-5642-4466-b395-787b5fecc3c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:49.987193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983b1dd-5642-4466-b395-787b5fecc3c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:49.988577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983b1dd-5642-4466-b395-787b5fecc3c2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:49.997803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 180/352] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:49.998878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125acdd8-0612-4aaa-ad1e-e966e4a4ed9c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:49.999160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125acdd8-0612-4aaa-ad1e-e966e4a4ed9c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:49.999434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125acdd8-0612-4aaa-ad1e-e966e4a4ed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.018068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-125acdd8-0612-4aaa-ad1e-e966e4a4ed9c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:50.018444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5deac-d013-4312-ad2e-e3c3d69a6bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.019285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-125acdd8-0612-4aaa-ad1e-e966e4a4ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:50.019568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5deac-d013-4312-ad2e-e3c3d69a6bde None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.020032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5deac-d013-4312-ad2e-e3c3d69a6bde None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.020486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5deac-d013-4312-ad2e-e3c3d69a6bde None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.022098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd5deac-d013-4312-ad2e-e3c3d69a6bde None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:50.023672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd5deac-d013-4312-ad2e-e3c3d69a6bde None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:50.026144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 173/353] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:49 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:50.027223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5deac-d013-4312-ad2e-e3c3d69a6bde None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.112748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 181/354] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:50.114608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125acdd8-0612-4aaa-ad1e-e966e4a4ed9c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.180843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:50.198029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5deac-d013-4312-ad2e-e3c3d69a6bde None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:50.233609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5859bcb-98b1-405a-a77f-974057e74bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.234155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5859bcb-98b1-405a-a77f-974057e74bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.234427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5859bcb-98b1-405a-a77f-974057e74bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.234692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5859bcb-98b1-405a-a77f-974057e74bc9 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.236734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125acdd8-0612-4aaa-ad1e-e966e4a4ed9c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:50.241543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5859bcb-98b1-405a-a77f-974057e74bc9 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:50.242426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5859bcb-98b1-405a-a77f-974057e74bc9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:50.244622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 174/355] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:50.245403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5859bcb-98b1-405a-a77f-974057e74bc9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.287212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:50.294479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7b524-95dc-4335-a4ae-2ec5e1ec7a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.295465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7b524-95dc-4335-a4ae-2ec5e1ec7a69 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.295993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7b524-95dc-4335-a4ae-2ec5e1ec7a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.296729 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7b524-95dc-4335-a4ae-2ec5e1ec7a69 None admin] PATH_INFO: `/v3/users/747e5f18d70e443193439fdef9d0c6b3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.299048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7b524-95dc-4335-a4ae-2ec5e1ec7a69 None admin] RBAC: Authorizing `identity:get_user(user_id=747e5f18d70e443193439fdef9d0c6b3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:50.301228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7b524-95dc-4335-a4ae-2ec5e1ec7a69 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:50.303539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 182/356] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/users/747e5f18d70e443193439fdef9d0c6b3 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:50.304546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7b524-95dc-4335-a4ae-2ec5e1ec7a69 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.353127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5859bcb-98b1-405a-a77f-974057e74bc9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:50.404870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8170f-13e5-4bba-bc38-ffa1a9e8aba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.405510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8170f-13e5-4bba-bc38-ffa1a9e8aba4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.405787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8170f-13e5-4bba-bc38-ffa1a9e8aba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.406639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8170f-13e5-4bba-bc38-ffa1a9e8aba4 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.408217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8170f-13e5-4bba-bc38-ffa1a9e8aba4 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:50.410640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8170f-13e5-4bba-bc38-ffa1a9e8aba4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:50.412488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 175/357] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:50.413072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8170f-13e5-4bba-bc38-ffa1a9e8aba4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.445735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7b524-95dc-4335-a4ae-2ec5e1ec7a69 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:50.474848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:50.485326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4b437-763f-4ee2-9ec9-d5f6b2fefe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.485896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4b437-763f-4ee2-9ec9-d5f6b2fefe6d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.486170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4b437-763f-4ee2-9ec9-d5f6b2fefe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.486437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4b437-763f-4ee2-9ec9-d5f6b2fefe6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.498254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4b437-763f-4ee2-9ec9-d5f6b2fefe6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-86f4b437-763f-4ee2-9ec9-d5f6b2fefe6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:50.502637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:50.503638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:50.504423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:50.507121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 183/358] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:50.507121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85578186-b6b9-4429-a89f-c9050df4d16f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.507121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85578186-b6b9-4429-a89f-c9050df4d16f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.507121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85578186-b6b9-4429-a89f-c9050df4d16f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.509293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 184/359] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:50.509293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85578186-b6b9-4429-a89f-c9050df4d16f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.533629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8170f-13e5-4bba-bc38-ffa1a9e8aba4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:50.541808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:50.586491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.587303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.587672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.587819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.588559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] RBAC: Adding query filter params (group.id=9b637c99470c4db3a99e64dfef03e668, role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:50.588860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:50.590311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:50.599367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 176/360] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/role_assignments?group.id=9b637c99470c4db3a99e64dfef03e668&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=d06115dbab78488182bcb432bf9bec11 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:50.604313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.642052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85578186-b6b9-4429-a89f-c9050df4d16f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:50.690598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.691604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.691876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.692141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.692883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] RBAC: Adding query filter params (role.id=28923fdfbb364d71a285386fd05d9fb2, user.id=747e5f18d70e443193439fdef9d0c6b3, scope.system=all) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:50.693297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:50.694811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:50.713857 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 185/361] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/role_assignments?user.id=747e5f18d70e443193439fdef9d0c6b3&scope.system=all&role.id=28923fdfbb364d71a285386fd05d9fb2 => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:50.716253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.751879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2f12e-b1ef-4430-8d5b-8cad754b8603 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:50.775176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:50.800363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.801416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.801893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.802342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.803302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] RBAC: Adding query filter params (group.id=9b637c99470c4db3a99e64dfef03e668, role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:50.803904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:50.805702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:50.818856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 177/362] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/role_assignments?group.id=9b637c99470c4db3a99e64dfef03e668&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=5bbcda16701c451f9419aefc4671840e => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:50.819909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:50.844057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac4cde-8f58-483c-a4b0-621b3f1d7207 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:50.864924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:50.884055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb4c8b-3278-419a-be10-19e13ae75c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:50.884866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb4c8b-3278-419a-be10-19e13ae75c42 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.885150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb4c8b-3278-419a-be10-19e13ae75c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.885474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb4c8b-3278-419a-be10-19e13ae75c42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.897940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb4c8b-3278-419a-be10-19e13ae75c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ccfb4c8b-3278-419a-be10-19e13ae75c42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:50.903633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:50.904557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:50.905345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 186/363] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:50.911257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c535c-ca00-452a-aeb5-c72104ecc8cc None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:50.911541 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c535c-ca00-452a-aeb5-c72104ecc8cc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:50.911820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c535c-ca00-452a-aeb5-c72104ecc8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:50.932130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-686c535c-ca00-452a-aeb5-c72104ecc8cc None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:50.933338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-686c535c-ca00-452a-aeb5-c72104ecc8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:50.963448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3e38-f4f5-452d-ab42-78dfef6b27db None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:51.008851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:51.009385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:51.009696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:51.010062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:51.010782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:51.011404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:51.013152 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:51.020604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 187/364] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:51.022474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c535c-ca00-452a-aeb5-c72104ecc8cc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:51.026792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 178/365] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:51.032700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:51.108208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:51.189270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c535c-ca00-452a-aeb5-c72104ecc8cc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:51.250291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:51.256418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:51.257904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:51.258547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:51.259111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:51.261339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537a286-a509-4e4e-b9db-adc7cc13cf4e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:51.262252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:51.262806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:51.264409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:51.283877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 188/366] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:51.289662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:51.329576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e00d6-f739-46fa-8954-f19716bbc105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:51.330560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e00d6-f739-46fa-8954-f19716bbc105 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:51.331095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e00d6-f739-46fa-8954-f19716bbc105 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:51.331558 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e00d6-f739-46fa-8954-f19716bbc105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:51.439757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e00d6-f739-46fa-8954-f19716bbc105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:51.442137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e00d6-f739-46fa-8954-f19716bbc105 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:51.456990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:51.463671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823cb7e-25ba-4d3b-8623-baf3fcd0929d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:51.505325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b50ab-5dc0-4ea4-86b2-96d93f8e3ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:51.506047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b50ab-5dc0-4ea4-86b2-96d93f8e3ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:51.506400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b50ab-5dc0-4ea4-86b2-96d93f8e3ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:51.506761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b50ab-5dc0-4ea4-86b2-96d93f8e3ef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:51.572945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 179/367] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:51.574191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e00d6-f739-46fa-8954-f19716bbc105 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:51.575084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:51.593813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b50ab-5dc0-4ea4-86b2-96d93f8e3ef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:51.595652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b50ab-5dc0-4ea4-86b2-96d93f8e3ef8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:51.713928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e00d6-f739-46fa-8954-f19716bbc105 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:51.724801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:51.730951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 189/368] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 04:21:51.731960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b50ab-5dc0-4ea4-86b2-96d93f8e3ef8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:51.768811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6fd93-5e1e-40fa-8acc-8598ca5f276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:51.769454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6fd93-5e1e-40fa-8acc-8598ca5f276f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:51.769735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6fd93-5e1e-40fa-8acc-8598ca5f276f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:51.770006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6fd93-5e1e-40fa-8acc-8598ca5f276f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:51.787562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6fd93-5e1e-40fa-8acc-8598ca5f276f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-28e6fd93-5e1e-40fa-8acc-8598ca5f276f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:51.793666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:51.794595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:51.795404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 180/369] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:51.796044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6fd93-5e1e-40fa-8acc-8598ca5f276f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:51.872910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b50ab-5dc0-4ea4-86b2-96d93f8e3ef8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:51.874163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:51.911889 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702248b4-40f7-4abd-af1c-5b8189ee517d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:51.912551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702248b4-40f7-4abd-af1c-5b8189ee517d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:51.912799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702248b4-40f7-4abd-af1c-5b8189ee517d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:51.913070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702248b4-40f7-4abd-af1c-5b8189ee517d None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:51.914292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702248b4-40f7-4abd-af1c-5b8189ee517d None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:51.916242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702248b4-40f7-4abd-af1c-5b8189ee517d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:51.918202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 190/370] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:51 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:51.921677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702248b4-40f7-4abd-af1c-5b8189ee517d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:51.935543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6fd93-5e1e-40fa-8acc-8598ca5f276f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:51.985571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5af7c-a163-4280-9b74-620b96ee523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:51.986375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5af7c-a163-4280-9b74-620b96ee523b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:51.986475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5af7c-a163-4280-9b74-620b96ee523b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:51.986757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5af7c-a163-4280-9b74-620b96ee523b None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:51.988190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5af7c-a163-4280-9b74-620b96ee523b None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:51.989868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5af7c-a163-4280-9b74-620b96ee523b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:51.991851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 181/371] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:51 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:51.992623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5af7c-a163-4280-9b74-620b96ee523b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:52.005137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:52.051795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702248b4-40f7-4abd-af1c-5b8189ee517d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:52.102805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:52.103854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:52.104353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:52.104812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:52.105755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:52.106318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:52.108145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:52.116142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5af7c-a163-4280-9b74-620b96ee523b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:52.119772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 191/372] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:52.120839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:52.144981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:52.157277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5ac4f-cfe5-42b9-a2c1-ddc48bfbf74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:52.158054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5ac4f-cfe5-42b9-a2c1-ddc48bfbf74c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:52.158351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5ac4f-cfe5-42b9-a2c1-ddc48bfbf74c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:52.158751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5ac4f-cfe5-42b9-a2c1-ddc48bfbf74c None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:52.159954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5ac4f-cfe5-42b9-a2c1-ddc48bfbf74c None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:52.162117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5ac4f-cfe5-42b9-a2c1-ddc48bfbf74c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:52.169026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 182/373] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:51 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:52.169026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5ac4f-cfe5-42b9-a2c1-ddc48bfbf74c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:52.273709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9450f-87ee-45fd-bda0-06a40d77820b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:52.297035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5ac4f-cfe5-42b9-a2c1-ddc48bfbf74c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:52.305299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:52.317408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce923f-b094-4155-8aad-bbe1f05ae2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:52.318058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce923f-b094-4155-8aad-bbe1f05ae2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:52.318341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce923f-b094-4155-8aad-bbe1f05ae2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:52.318629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce923f-b094-4155-8aad-bbe1f05ae2f2 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:52.319950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce923f-b094-4155-8aad-bbe1f05ae2f2 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:52.321653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce923f-b094-4155-8aad-bbe1f05ae2f2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:52.324529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 192/374] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:52 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:52.324650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce923f-b094-4155-8aad-bbe1f05ae2f2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:52.345864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975af0ec-3bf2-42b3-8f55-e14ab3cc6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:52.346902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975af0ec-3bf2-42b3-8f55-e14ab3cc6468 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:52.347467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975af0ec-3bf2-42b3-8f55-e14ab3cc6468 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:52.347949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975af0ec-3bf2-42b3-8f55-e14ab3cc6468 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:52.416584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:52.430996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975af0ec-3bf2-42b3-8f55-e14ab3cc6468 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:52.432813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975af0ec-3bf2-42b3-8f55-e14ab3cc6468 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:52.452313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce923f-b094-4155-8aad-bbe1f05ae2f2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:52.491141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:52.492118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:52.492677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:52.493137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:52.494166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] RBAC: Adding query filter params (group.id=9b637c99470c4db3a99e64dfef03e668, role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=a33c93942a544564b55765288214ed83) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:52.494708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:52.496607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:52.503009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 193/375] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:21:52 2022] GET /identity/v3/role_assignments?group.id=9b637c99470c4db3a99e64dfef03e668&scope.project.id=a33c93942a544564b55765288214ed83&role.id=d06115dbab78488182bcb432bf9bec11 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:52.504136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:52.541825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 183/376] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:21:52.547683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975af0ec-3bf2-42b3-8f55-e14ab3cc6468 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:52.572438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:52.625653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812b64b-03d0-4c26-b908-fe69731f4786 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:52.669106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:52.670289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:52.670816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:52.671279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:52.672227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] RBAC: Adding query filter params (group.id=9b637c99470c4db3a99e64dfef03e668, role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=a33c93942a544564b55765288214ed83) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:52.672731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:52.674400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:52.682682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 194/377] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:21:52 2022] GET /identity/v3/role_assignments?group.id=9b637c99470c4db3a99e64dfef03e668&scope.project.id=a33c93942a544564b55765288214ed83&role.id=5bbcda16701c451f9419aefc4671840e => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:52.744624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:52.756239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975af0ec-3bf2-42b3-8f55-e14ab3cc6468 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:52.802495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c41329-bf15-44a0-8565-1d8f7a336b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:52.803207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c41329-bf15-44a0-8565-1d8f7a336b5e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:52.803489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c41329-bf15-44a0-8565-1d8f7a336b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:52.803762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c41329-bf15-44a0-8565-1d8f7a336b5e None admin] PATH_INFO: `/v3/projects/529acf6d876d441682b1d4508345bc61` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:52.804994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c41329-bf15-44a0-8565-1d8f7a336b5e None admin] RBAC: Authorizing `identity:get_project(project_id=529acf6d876d441682b1d4508345bc61)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:52.806652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c41329-bf15-44a0-8565-1d8f7a336b5e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:52.808447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 184/378] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:52 2022] GET /identity/v3/projects/529acf6d876d441682b1d4508345bc61 => generated 344 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:52.814261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:52.955053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388d50-a883-4fc3-93ec-d162b364a87f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:52.981289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:52.997838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4674-f3ae-4ad8-bea2-563e09c5e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:52.998913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4674-f3ae-4ad8-bea2-563e09c5e69d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:52.999387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4674-f3ae-4ad8-bea2-563e09c5e69d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:52.999877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4674-f3ae-4ad8-bea2-563e09c5e69d None admin] PATH_INFO: `/v3/groups/b5ebac73de0c451c914c4f7224d20f23` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.009425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4674-f3ae-4ad8-bea2-563e09c5e69d None admin] RBAC: Authorizing `identity:get_group(group_id=b5ebac73de0c451c914c4f7224d20f23)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:53.010454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006858-d04a-4877-a2e3-cdbaf27b92a9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:53.010939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006858-d04a-4877-a2e3-cdbaf27b92a9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:53.011134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006858-d04a-4877-a2e3-cdbaf27b92a9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.012199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4674-f3ae-4ad8-bea2-563e09c5e69d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:53.012496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 185/379] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:53.014653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 195/380] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:52 2022] GET /identity/v3/groups/b5ebac73de0c451c914c4f7224d20f23 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:53.019947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4674-f3ae-4ad8-bea2-563e09c5e69d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:53.020281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3163-6cb1-47e9-982e-7bbddd8595ac None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:53.020560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3163-6cb1-47e9-982e-7bbddd8595ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:53.020833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3163-6cb1-47e9-982e-7bbddd8595ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.045071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c3b3163-6cb1-47e9-982e-7bbddd8595ac None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:53.046383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c3b3163-6cb1-47e9-982e-7bbddd8595ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:53.173631 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 186/381] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:53.175703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6859515-b97b-47f7-8efd-cf344dbce80a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:53.176117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6859515-b97b-47f7-8efd-cf344dbce80a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:53.176387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6859515-b97b-47f7-8efd-cf344dbce80a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.178579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 187/382] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:53.183828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c428af-6df8-4be8-a128-6e0fcbd5da8c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:53.184092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c428af-6df8-4be8-a128-6e0fcbd5da8c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:53.184356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c428af-6df8-4be8-a128-6e0fcbd5da8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.192409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4674-f3ae-4ad8-bea2-563e09c5e69d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:53.204867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d4c428af-6df8-4be8-a128-6e0fcbd5da8c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:53.206403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d4c428af-6df8-4be8-a128-6e0fcbd5da8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:53.229425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:53.230045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:53.230324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:53.230597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.231366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] RBAC: Adding query filter params (group.id=b5ebac73de0c451c914c4f7224d20f23, role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=529acf6d876d441682b1d4508345bc61) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:53.231674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:53.233215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:53.239333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 196/383] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:21:53 2022] GET /identity/v3/role_assignments?group.id=b5ebac73de0c451c914c4f7224d20f23&scope.project.id=529acf6d876d441682b1d4508345bc61&role.id=241e99d365564c0fa95cb712f4004785 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:53.288621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 188/384] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:53.577978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:53.631985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:53.673280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c428af-6df8-4be8-a128-6e0fcbd5da8c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:53.684037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d81a4-1419-4135-ac2e-c4f9e754204c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:53.713643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:53.719418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d9a76-caa1-4616-83ec-bb0112a8beec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:53.720057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9a76-caa1-4616-83ec-bb0112a8beec None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:53.720337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9a76-caa1-4616-83ec-bb0112a8beec None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:53.720607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9a76-caa1-4616-83ec-bb0112a8beec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.735938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d9a76-caa1-4616-83ec-bb0112a8beec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-954d9a76-caa1-4616-83ec-bb0112a8beec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:53.741843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:53.742741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:53.743515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:53.749431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 197/385] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:53.752072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d9a76-caa1-4616-83ec-bb0112a8beec None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:53.863563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c428af-6df8-4be8-a128-6e0fcbd5da8c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:53.870494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d9a76-caa1-4616-83ec-bb0112a8beec None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:53.879526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:53.912056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:53.912603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:53.912889 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:53.913165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.913843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:53.914144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:53.924017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:53.935656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 198/386] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:53.944766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:53.945762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2397b2-22c8-4a64-abb8-45a22c6be68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:53.946749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2397b2-22c8-4a64-abb8-45a22c6be68a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:53.947355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2397b2-22c8-4a64-abb8-45a22c6be68a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:53.947947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2397b2-22c8-4a64-abb8-45a22c6be68a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:53.980965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2397b2-22c8-4a64-abb8-45a22c6be68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1c2397b2-22c8-4a64-abb8-45a22c6be68a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:53.990740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:53.991640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:53.992460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:53.996725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 189/387] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:54.001800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2397b2-22c8-4a64-abb8-45a22c6be68a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:54.113514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7236d86-c010-495c-84d1-b68642907007 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:54.169738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb8d5c-8964-43b6-8f5c-b5bc819a5a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:54.170317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8d5c-8964-43b6-8f5c-b5bc819a5a45 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:54.170599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8d5c-8964-43b6-8f5c-b5bc819a5a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:54.171364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8d5c-8964-43b6-8f5c-b5bc819a5a45 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:54.172588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb8d5c-8964-43b6-8f5c-b5bc819a5a45 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:54.174064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb8d5c-8964-43b6-8f5c-b5bc819a5a45 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:54.182891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 199/388] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:53 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:54.184622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb8d5c-8964-43b6-8f5c-b5bc819a5a45 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:54.203087 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:54.253681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:54.259749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2397b2-22c8-4a64-abb8-45a22c6be68a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:54.354890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb8d5c-8964-43b6-8f5c-b5bc819a5a45 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:54.367673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:54.368760 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:54.369288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:54.369754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:54.370826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:54.371335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:54.373111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:54.394369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 190/389] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:54.399559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:54.420684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8ca5b-5d04-468f-8828-99f054166afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:54.421315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8ca5b-5d04-468f-8828-99f054166afa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:54.426740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8ca5b-5d04-468f-8828-99f054166afa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:54.426870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8ca5b-5d04-468f-8828-99f054166afa None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:54.428148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8ca5b-5d04-468f-8828-99f054166afa None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:54.429617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8ca5b-5d04-468f-8828-99f054166afa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:54.434616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 200/390] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:54 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:54.445375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8ca5b-5d04-468f-8828-99f054166afa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:54.626761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8ca5b-5d04-468f-8828-99f054166afa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:54.631796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1eb307-6db2-40c8-a6a0-7d86cdbfaae9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:54.638232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:54.674819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ff853-6d1d-4b56-9a98-1369b4a7b1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:54.674819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff853-6d1d-4b56-9a98-1369b4a7b1cc None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:54.674819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff853-6d1d-4b56-9a98-1369b4a7b1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:54.674819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff853-6d1d-4b56-9a98-1369b4a7b1cc None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/groups/9b637c99470c4db3a99e64dfef03e668/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:54.675252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ff853-6d1d-4b56-9a98-1369b4a7b1cc None admin] RBAC: Authorizing `identity:create_grant(project_id=5db3d2a60ec94cf1ad027cdaa14910bf, group_id=9b637c99470c4db3a99e64dfef03e668, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:54.675502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ff853-6d1d-4b56-9a98-1369b4a7b1cc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:54.679402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8b97c-f64d-4bcb-abb9-5fe44299d8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:54.680214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8b97c-f64d-4bcb-abb9-5fe44299d8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:54.680614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8b97c-f64d-4bcb-abb9-5fe44299d8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:54.681013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8b97c-f64d-4bcb-abb9-5fe44299d8d7 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:54.687423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8b97c-f64d-4bcb-abb9-5fe44299d8d7 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:54.689221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8b97c-f64d-4bcb-abb9-5fe44299d8d7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:54.691838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 191/391] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:54 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:54.694094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 201/392] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 31 04:21:54 2022] PUT /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/groups/9b637c99470c4db3a99e64dfef03e668/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:54.698228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8b97c-f64d-4bcb-abb9-5fe44299d8d7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:54.728041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:54.834928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8b97c-f64d-4bcb-abb9-5fe44299d8d7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:54.874281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67044466-0724-4cf2-9099-4ec624db193e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:54.874970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67044466-0724-4cf2-9099-4ec624db193e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:54.875251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67044466-0724-4cf2-9099-4ec624db193e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:54.875527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67044466-0724-4cf2-9099-4ec624db193e None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:54.876693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67044466-0724-4cf2-9099-4ec624db193e None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:54.878230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67044466-0724-4cf2-9099-4ec624db193e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:54.880293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 192/393] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:54 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:54.886032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ff853-6d1d-4b56-9a98-1369b4a7b1cc None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:54.933460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f62f-ebb0-4376-808a-4c864751a9c2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:54.933753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f62f-ebb0-4376-808a-4c864751a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:54.933991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f62f-ebb0-4376-808a-4c864751a9c2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:54.935552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 193/394] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:54.940516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ff8a4-c526-407d-a507-15cbcf646144 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:54.940800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ff8a4-c526-407d-a507-15cbcf646144 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:54.941073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ff8a4-c526-407d-a507-15cbcf646144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:54.961596 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e60ff8a4-c526-407d-a507-15cbcf646144 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:54.962848 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e60ff8a4-c526-407d-a507-15cbcf646144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:54.999153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ff853-6d1d-4b56-9a98-1369b4a7b1cc None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:55.037515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1bcdc-009b-4511-8331-239270149d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:55.038198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1bcdc-009b-4511-8331-239270149d41 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:55.038476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1bcdc-009b-4511-8331-239270149d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:55.038770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1bcdc-009b-4511-8331-239270149d41 None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/groups/9b637c99470c4db3a99e64dfef03e668/roles/d06115dbab78488182bcb432bf9bec11` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:55.041244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1bcdc-009b-4511-8331-239270149d41 None admin] RBAC: Authorizing `identity:create_grant(project_id=5db3d2a60ec94cf1ad027cdaa14910bf, group_id=9b637c99470c4db3a99e64dfef03e668, role_id=d06115dbab78488182bcb432bf9bec11)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:55.042748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1bcdc-009b-4511-8331-239270149d41 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:55.047132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 194/395] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:55.058910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 202/396] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 31 04:21:54 2022] PUT /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf/groups/9b637c99470c4db3a99e64dfef03e668/roles/d06115dbab78488182bcb432bf9bec11 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:55.339067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ff8a4-c526-407d-a507-15cbcf646144 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:55.350319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:55.407958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6accb31-debd-49fe-ba8a-fe55e38eba8f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:55.408487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6accb31-debd-49fe-ba8a-fe55e38eba8f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:55.408562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6accb31-debd-49fe-ba8a-fe55e38eba8f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:55.409967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 203/397] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:55.419128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812ebcb-9e24-4cb3-ac1a-fba6e0443f13 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:55.419492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812ebcb-9e24-4cb3-ac1a-fba6e0443f13 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:55.420110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812ebcb-9e24-4cb3-ac1a-fba6e0443f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:55.444715 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d812ebcb-9e24-4cb3-ac1a-fba6e0443f13 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:55.447744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d812ebcb-9e24-4cb3-ac1a-fba6e0443f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:55.457788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ff8a4-c526-407d-a507-15cbcf646144 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:55.483096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:55.521821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae198a34-6de1-4b39-8fd2-b8f015742b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:55.522464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae198a34-6de1-4b39-8fd2-b8f015742b51 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:55.522752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae198a34-6de1-4b39-8fd2-b8f015742b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:55.523023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae198a34-6de1-4b39-8fd2-b8f015742b51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:55.545271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae198a34-6de1-4b39-8fd2-b8f015742b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:55.552012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 204/398] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ae198a34-6de1-4b39-8fd2-b8f015742b51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:55.552520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:55.553346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:55.554442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 195/399] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:55.559594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812ebcb-9e24-4cb3-ac1a-fba6e0443f13 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:55.702478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812ebcb-9e24-4cb3-ac1a-fba6e0443f13 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:55.739643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:55.740260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:55.740525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:55.740792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:55.741461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:55.741759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:55.743177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:55.751989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 205/400] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:55.759833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae198a34-6de1-4b39-8fd2-b8f015742b51 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:55.841044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:55.863620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:55.892085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:55.908935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae198a34-6de1-4b39-8fd2-b8f015742b51 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:55.945319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da32089-5a90-4a69-8eb2-7d2f38762da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:55.945955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da32089-5a90-4a69-8eb2-7d2f38762da0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:55.946232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da32089-5a90-4a69-8eb2-7d2f38762da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:55.946507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da32089-5a90-4a69-8eb2-7d2f38762da0 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:55.947708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da32089-5a90-4a69-8eb2-7d2f38762da0 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:55.949151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da32089-5a90-4a69-8eb2-7d2f38762da0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:55.951062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 196/401] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:55 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:55.956417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da32089-5a90-4a69-8eb2-7d2f38762da0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:56.034270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13284031-02cf-45a0-96dc-e76ae994a40a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:56.099114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b177ce-e30c-4ff3-a9c8-766f08df855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:56.100141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b177ce-e30c-4ff3-a9c8-766f08df855b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.100616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b177ce-e30c-4ff3-a9c8-766f08df855b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.101075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b177ce-e30c-4ff3-a9c8-766f08df855b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.117620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b177ce-e30c-4ff3-a9c8-766f08df855b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a8b177ce-e30c-4ff3-a9c8-766f08df855b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:56.123595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:56.124451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:56.125241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:56.127910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 206/402] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:56.133699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b177ce-e30c-4ff3-a9c8-766f08df855b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:56.134409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da32089-5a90-4a69-8eb2-7d2f38762da0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:56.178065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50b69-6761-494d-8752-38bc25438f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:56.179128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50b69-6761-494d-8752-38bc25438f3d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.179869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50b69-6761-494d-8752-38bc25438f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.180346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50b69-6761-494d-8752-38bc25438f3d None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.182749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50b69-6761-494d-8752-38bc25438f3d None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:56.184417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50b69-6761-494d-8752-38bc25438f3d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:56.187093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 197/403] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:55 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:56.192375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac7558-00b5-4098-a728-ee8534645e27 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.192551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac7558-00b5-4098-a728-ee8534645e27 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.192810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac7558-00b5-4098-a728-ee8534645e27 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.195991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 198/404] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:56.197452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac7558-00b5-4098-a728-ee8534645e27 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:56.202140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:56.243693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:56.267780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b177ce-e30c-4ff3-a9c8-766f08df855b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:56.311286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc362c-47ba-46f9-a125-6b56d4c46733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:56.311907 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc362c-47ba-46f9-a125-6b56d4c46733 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.312191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc362c-47ba-46f9-a125-6b56d4c46733 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.312470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc362c-47ba-46f9-a125-6b56d4c46733 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.313142 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc362c-47ba-46f9-a125-6b56d4c46733 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:56.313447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc362c-47ba-46f9-a125-6b56d4c46733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:56.314975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc362c-47ba-46f9-a125-6b56d4c46733 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:56.326780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 207/405] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:56.326780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e9508-0628-456f-869a-f05313967675 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.326780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e9508-0628-456f-869a-f05313967675 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.326780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e9508-0628-456f-869a-f05313967675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.340235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac7558-00b5-4098-a728-ee8534645e27 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:56.348121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d46e9508-0628-456f-869a-f05313967675 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:56.349462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d46e9508-0628-456f-869a-f05313967675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:56.383669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41324-d421-43d7-98f3-6509bc308e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:56.384566 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41324-d421-43d7-98f3-6509bc308e68 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.384870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41324-d421-43d7-98f3-6509bc308e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.385138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41324-d421-43d7-98f3-6509bc308e68 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83/groups/9b637c99470c4db3a99e64dfef03e668/roles/d06115dbab78488182bcb432bf9bec11` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.387365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41324-d421-43d7-98f3-6509bc308e68 None admin] RBAC: Authorizing `identity:create_grant(project_id=a33c93942a544564b55765288214ed83, group_id=9b637c99470c4db3a99e64dfef03e668, role_id=d06115dbab78488182bcb432bf9bec11)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:56.389922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41324-d421-43d7-98f3-6509bc308e68 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:56.410810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 199/406] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 31 04:21:56 2022] PUT /identity/v3/projects/a33c93942a544564b55765288214ed83/groups/9b637c99470c4db3a99e64dfef03e668/roles/d06115dbab78488182bcb432bf9bec11 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:56.411892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41324-d421-43d7-98f3-6509bc308e68 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:56.447976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 208/407] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:56.511074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:56.563384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41324-d421-43d7-98f3-6509bc308e68 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:56.610036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a428d-63a2-4bc9-a528-0e477c458cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:56.611110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a428d-63a2-4bc9-a528-0e477c458cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.611601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a428d-63a2-4bc9-a528-0e477c458cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.612079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a428d-63a2-4bc9-a528-0e477c458cd6 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.613406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a428d-63a2-4bc9-a528-0e477c458cd6 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:56.615077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a428d-63a2-4bc9-a528-0e477c458cd6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:56.616843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 200/408] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:56 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:56.622497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e9508-0628-456f-869a-f05313967675 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:56.703065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:56.735651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e9508-0628-456f-869a-f05313967675 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:56.774229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954060-94c7-4c37-8af3-26e4e91c8302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:56.774825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954060-94c7-4c37-8af3-26e4e91c8302 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.775099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954060-94c7-4c37-8af3-26e4e91c8302 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.775449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954060-94c7-4c37-8af3-26e4e91c8302 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.777113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954060-94c7-4c37-8af3-26e4e91c8302 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:56.778372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954060-94c7-4c37-8af3-26e4e91c8302 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:56.780333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 209/409] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:56 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:56.786090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a428d-63a2-4bc9-a528-0e477c458cd6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:56.794452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954060-94c7-4c37-8af3-26e4e91c8302 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:56.928890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954060-94c7-4c37-8af3-26e4e91c8302 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:56.963791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78f607-d4d7-4cf2-a329-02ed6dfcc2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:56.964874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f607-d4d7-4cf2-a329-02ed6dfcc2fd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:56.965359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f607-d4d7-4cf2-a329-02ed6dfcc2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:56.965816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f607-d4d7-4cf2-a329-02ed6dfcc2fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:56.977058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78f607-d4d7-4cf2-a329-02ed6dfcc2fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-bf78f607-d4d7-4cf2-a329-02ed6dfcc2fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:56.982277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:56.983210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:21:56.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:56.986610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 210/410] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:21:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:21:56.986928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:56.998229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78f607-d4d7-4cf2-a329-02ed6dfcc2fd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:57.012679 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a428d-63a2-4bc9-a528-0e477c458cd6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:57.079090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:57.087467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e291a-3019-4d7d-81d1-bacdd0290052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:57.088157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e291a-3019-4d7d-81d1-bacdd0290052 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.088429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e291a-3019-4d7d-81d1-bacdd0290052 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.088692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e291a-3019-4d7d-81d1-bacdd0290052 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83/groups/9b637c99470c4db3a99e64dfef03e668/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.090863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e291a-3019-4d7d-81d1-bacdd0290052 None admin] RBAC: Authorizing `identity:create_grant(project_id=a33c93942a544564b55765288214ed83, group_id=9b637c99470c4db3a99e64dfef03e668, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:57.092356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e291a-3019-4d7d-81d1-bacdd0290052 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:57.116277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 201/411] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 31 04:21:56 2022] PUT /identity/v3/projects/a33c93942a544564b55765288214ed83/groups/9b637c99470c4db3a99e64dfef03e668/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:57.138654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a969c-6ae4-4979-ace9-f9af6ff01a23 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.139001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a969c-6ae4-4979-ace9-f9af6ff01a23 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.139254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a969c-6ae4-4979-ace9-f9af6ff01a23 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.140755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 202/412] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:57.146477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7ee3f-d521-4049-8a22-c920f02fb388 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.146816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7ee3f-d521-4049-8a22-c920f02fb388 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.147063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7ee3f-d521-4049-8a22-c920f02fb388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.173076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f3e7ee3f-d521-4049-8a22-c920f02fb388 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:57.174293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f3e7ee3f-d521-4049-8a22-c920f02fb388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:57.206064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78f607-d4d7-4cf2-a329-02ed6dfcc2fd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:57.255994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:57.257027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.257493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.257963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.258883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:57.259485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:57.261308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:57.271340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 211/413] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:57.282590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:57.303549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 203/414] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:57.356274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:57.396556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca405750-95cf-40de-8779-2f04d2cf9ae9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:57.430687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dee1a9-fe54-4911-8f96-b1c721450139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:57.431354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dee1a9-fe54-4911-8f96-b1c721450139 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.431746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dee1a9-fe54-4911-8f96-b1c721450139 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.432021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dee1a9-fe54-4911-8f96-b1c721450139 None admin] PATH_INFO: `/v3/groups/b5ebac73de0c451c914c4f7224d20f23` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.433216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dee1a9-fe54-4911-8f96-b1c721450139 None admin] RBAC: Authorizing `identity:get_group(group_id=b5ebac73de0c451c914c4f7224d20f23)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:57.434658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dee1a9-fe54-4911-8f96-b1c721450139 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:57.436566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 212/415] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:57 2022] GET /identity/v3/groups/b5ebac73de0c451c914c4f7224d20f23 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:57.442192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7ee3f-d521-4049-8a22-c920f02fb388 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:57.575577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7ee3f-d521-4049-8a22-c920f02fb388 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:57.592009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0a3f-2d31-4500-8bcf-e8314f9de9c3 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.592514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0a3f-2d31-4500-8bcf-e8314f9de9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.592729 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0a3f-2d31-4500-8bcf-e8314f9de9c3 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.594158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 213/416] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:21:57.594704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba0a3f-2d31-4500-8bcf-e8314f9de9c3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:57.599307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:57.612611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581993c-b405-44ad-9866-ad28e6ef58ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:57.613174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581993c-b405-44ad-9866-ad28e6ef58ca None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.613447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581993c-b405-44ad-9866-ad28e6ef58ca None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.613715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581993c-b405-44ad-9866-ad28e6ef58ca None admin] PATH_INFO: `/v3/projects/529acf6d876d441682b1d4508345bc61` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.614852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581993c-b405-44ad-9866-ad28e6ef58ca None admin] RBAC: Authorizing `identity:get_project(project_id=529acf6d876d441682b1d4508345bc61)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:57.616300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581993c-b405-44ad-9866-ad28e6ef58ca None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:57.618078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 204/417] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:57 2022] GET /identity/v3/projects/529acf6d876d441682b1d4508345bc61 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:57.622525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f83b1-9d43-4399-b9d9-365cfb5688e9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.622525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f83b1-9d43-4399-b9d9-365cfb5688e9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.622525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f83b1-9d43-4399-b9d9-365cfb5688e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.638491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eb1f83b1-9d43-4399-b9d9-365cfb5688e9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:57.639708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb1f83b1-9d43-4399-b9d9-365cfb5688e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:57.717793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 205/418] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:57.719300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f83b1-9d43-4399-b9d9-365cfb5688e9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:57.743354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba0a3f-2d31-4500-8bcf-e8314f9de9c3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:57.774407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:57.780704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9229a-364a-4882-8b1e-5a8808b3b9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:57.781280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9229a-364a-4882-8b1e-5a8808b3b9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.781558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9229a-364a-4882-8b1e-5a8808b3b9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.781989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9229a-364a-4882-8b1e-5a8808b3b9e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.794641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9229a-364a-4882-8b1e-5a8808b3b9e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-5ca9229a-364a-4882-8b1e-5a8808b3b9e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:57.799553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:57.800424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:57.801444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 214/419] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:21:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:57.809498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9229a-364a-4882-8b1e-5a8808b3b9e9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:57.889694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f83b1-9d43-4399-b9d9-365cfb5688e9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:57.912387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:57.928234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c5c7-06bf-4125-8947-1d4ac80f6f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:57.928900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c5c7-06bf-4125-8947-1d4ac80f6f85 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:57.929172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c5c7-06bf-4125-8947-1d4ac80f6f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:57.929440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c5c7-06bf-4125-8947-1d4ac80f6f85 None admin] PATH_INFO: `/v3/projects/529acf6d876d441682b1d4508345bc61/groups/b5ebac73de0c451c914c4f7224d20f23/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:57.931313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3c5c7-06bf-4125-8947-1d4ac80f6f85 None admin] RBAC: Authorizing `identity:create_grant(project_id=529acf6d876d441682b1d4508345bc61, group_id=b5ebac73de0c451c914c4f7224d20f23, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:57.933009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3c5c7-06bf-4125-8947-1d4ac80f6f85 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:57.952434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 206/420] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Aug 31 04:21:57 2022] PUT /identity/v3/projects/529acf6d876d441682b1d4508345bc61/groups/b5ebac73de0c451c914c4f7224d20f23/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:21:58.002084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9229a-364a-4882-8b1e-5a8808b3b9e9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:58.050876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:58.051508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.051784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.052051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:58.052733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:58.053032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:58.054510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:58.064327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c5c7-06bf-4125-8947-1d4ac80f6f85 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:58.071446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 215/421] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:21:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:58.077117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:58.142646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:58.184934 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c5c7-06bf-4125-8947-1d4ac80f6f85 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:58.209767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37520db6-1e07-48a2-9f49-ad0bd9b731fd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:58.227609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b913-d67e-4f4f-b7fd-b70a39ace82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:58.228998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b913-d67e-4f4f-b7fd-b70a39ace82e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.229469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b913-d67e-4f4f-b7fd-b70a39ace82e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.229920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b913-d67e-4f4f-b7fd-b70a39ace82e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:58.241622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514b913-d67e-4f4f-b7fd-b70a39ace82e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e514b913-d67e-4f4f-b7fd-b70a39ace82e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:58.247276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:58.248117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:58.248903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:21:58.250731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bd7e0-91a7-411f-9127-c758549cf86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:58.251588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd7e0-91a7-411f-9127-c758549cf86e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.251705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd7e0-91a7-411f-9127-c758549cf86e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.251908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd7e0-91a7-411f-9127-c758549cf86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:58.256092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 207/422] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:58.258710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b913-d67e-4f4f-b7fd-b70a39ace82e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:58.315103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:58.332721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bd7e0-91a7-411f-9127-c758549cf86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:58.334524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bd7e0-91a7-411f-9127-c758549cf86e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:58.399453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:58.402944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b913-d67e-4f4f-b7fd-b70a39ace82e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:58.445550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 216/423] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 04:21:58.451793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bd7e0-91a7-411f-9127-c758549cf86e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:58.478039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:58.479011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.479289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.479555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:58.480222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:58.480516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:58.498918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:58.526845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 208/424] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:58.536193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:58.622750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bd7e0-91a7-411f-9127-c758549cf86e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:58.666640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ccd75-4fd4-4cb7-81aa-578617e04fd3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:58.671036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:58.705753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97256fb-2728-4a0e-bd80-c0dc3478ab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:58.706997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97256fb-2728-4a0e-bd80-c0dc3478ab02 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.707525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97256fb-2728-4a0e-bd80-c0dc3478ab02 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.707813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25d3d-39aa-4add-89f3-ac932f7f533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:58.708222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97256fb-2728-4a0e-bd80-c0dc3478ab02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:58.708454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25d3d-39aa-4add-89f3-ac932f7f533e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.708730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25d3d-39aa-4add-89f3-ac932f7f533e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.708998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25d3d-39aa-4add-89f3-ac932f7f533e None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:58.710151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c25d3d-39aa-4add-89f3-ac932f7f533e None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:58.719748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c25d3d-39aa-4add-89f3-ac932f7f533e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:58.721778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 217/425] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:58 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:58.726883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25d3d-39aa-4add-89f3-ac932f7f533e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:58.785819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97256fb-2728-4a0e-bd80-c0dc3478ab02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:58.787959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97256fb-2728-4a0e-bd80-c0dc3478ab02 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:58.793834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:58.911782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 209/426] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 04:21:58.912166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ef774-d2fe-4cac-9b54-aaefddae131a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.912452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ef774-d2fe-4cac-9b54-aaefddae131a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.912825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ef774-d2fe-4cac-9b54-aaefddae131a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:58.915867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 210/427] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:58.916283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ef774-d2fe-4cac-9b54-aaefddae131a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:58.919366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:58.926050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25d3d-39aa-4add-89f3-ac932f7f533e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:58.968182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5bbdd-082a-474b-a2db-aa8fae8cc69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:58.968819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5bbdd-082a-474b-a2db-aa8fae8cc69c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.969102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5bbdd-082a-474b-a2db-aa8fae8cc69c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.969373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5bbdd-082a-474b-a2db-aa8fae8cc69c None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:58.970496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5bbdd-082a-474b-a2db-aa8fae8cc69c None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:58.972000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5bbdd-082a-474b-a2db-aa8fae8cc69c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:58.978782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 218/428] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:58 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:58.981772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a8656-1b55-4d73-bce2-7552af0644a8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:58.982044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a8656-1b55-4d73-bce2-7552af0644a8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:58.982307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a8656-1b55-4d73-bce2-7552af0644a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.001480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ad8a8656-1b55-4d73-bce2-7552af0644a8 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:59.002782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ad8a8656-1b55-4d73-bce2-7552af0644a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:59.042470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ef774-d2fe-4cac-9b54-aaefddae131a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:59.079267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 219/429] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:21:59.080331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a8656-1b55-4d73-bce2-7552af0644a8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:59.087778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef37d65-d03e-472a-bc10-aa9b43113a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:59.089046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef37d65-d03e-472a-bc10-aa9b43113a9a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:59.089551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef37d65-d03e-472a-bc10-aa9b43113a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:59.090013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef37d65-d03e-472a-bc10-aa9b43113a9a None admin] PATH_INFO: `/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.091394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef37d65-d03e-472a-bc10-aa9b43113a9a None admin] RBAC: Authorizing `identity:get_project(project_id=5db3d2a60ec94cf1ad027cdaa14910bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:59.093162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef37d65-d03e-472a-bc10-aa9b43113a9a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:59.095231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 211/430] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:21:58 2022] GET /identity/v3/projects/5db3d2a60ec94cf1ad027cdaa14910bf => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:59.106734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef37d65-d03e-472a-bc10-aa9b43113a9a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:59.107005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:59.180071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:59.213384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef37d65-d03e-472a-bc10-aa9b43113a9a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:59.255764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd46a2-7a21-41f8-ad64-47e09ea02662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:59.256832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd46a2-7a21-41f8-ad64-47e09ea02662 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:59.257333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd46a2-7a21-41f8-ad64-47e09ea02662 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:59.257839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd46a2-7a21-41f8-ad64-47e09ea02662 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.259325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd46a2-7a21-41f8-ad64-47e09ea02662 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:59.261184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd46a2-7a21-41f8-ad64-47e09ea02662 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:59.265935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 212/431] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:21:59 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:59.271065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd46a2-7a21-41f8-ad64-47e09ea02662 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:59.299692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a8656-1b55-4d73-bce2-7552af0644a8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:59.375950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:59.376961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:59.377452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:59.377931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.378887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] RBAC: Adding query filter params (group.id=9b637c99470c4db3a99e64dfef03e668, role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:59.379445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:59.381095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:59.393451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 220/432] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:21:59 2022] GET /identity/v3/role_assignments?group.id=9b637c99470c4db3a99e64dfef03e668&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=5bbcda16701c451f9419aefc4671840e => generated 611 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:59.419244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:59.461209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd46a2-7a21-41f8-ad64-47e09ea02662 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:59.534563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758852b1-2707-4bfa-9b65-84a20f3e393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:59.536060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758852b1-2707-4bfa-9b65-84a20f3e393d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:59.536550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758852b1-2707-4bfa-9b65-84a20f3e393d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:59.537003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758852b1-2707-4bfa-9b65-84a20f3e393d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.537926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758852b1-2707-4bfa-9b65-84a20f3e393d None admin] RBAC: Adding query filter params (group.id=9b637c99470c4db3a99e64dfef03e668, role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:59.538418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758852b1-2707-4bfa-9b65-84a20f3e393d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:59.540128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758852b1-2707-4bfa-9b65-84a20f3e393d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:59.560183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 213/433] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:21:59 2022] GET /identity/v3/role_assignments?group.id=9b637c99470c4db3a99e64dfef03e668&scope.project.id=5db3d2a60ec94cf1ad027cdaa14910bf&role.id=d06115dbab78488182bcb432bf9bec11 => generated 611 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:21:59.584315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0585f22-c5cd-41ce-b582-aa87de43d59a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:59.586688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:59.632083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68bc64-dfb6-4a74-805f-0bcfad235256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:59.633584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68bc64-dfb6-4a74-805f-0bcfad235256 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:59.634079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68bc64-dfb6-4a74-805f-0bcfad235256 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:59.634557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68bc64-dfb6-4a74-805f-0bcfad235256 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.648119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68bc64-dfb6-4a74-805f-0bcfad235256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:21:59.651146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf16d7-ec3f-4762-baa1-27fc9d5a9518 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:59.651428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf16d7-ec3f-4762-baa1-27fc9d5a9518 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:59.651694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf16d7-ec3f-4762-baa1-27fc9d5a9518 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-6d68bc64-dfb6-4a74-805f-0bcfad235256 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:21:59.654804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:21:59.655660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:21:59.656444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:21:59.657110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 214/434] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:21:59.658213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9592f-ef51-4c73-b33d-ba52140ce158 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:59.658488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9592f-ef51-4c73-b33d-ba52140ce158 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:59.658805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9592f-ef51-4c73-b33d-ba52140ce158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.659967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 221/435] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:21:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:21:59.665699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68bc64-dfb6-4a74-805f-0bcfad235256 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:59.667903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:59.682162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b4e9592f-ef51-4c73-b33d-ba52140ce158 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:21:59.683514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b4e9592f-ef51-4c73-b33d-ba52140ce158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:21:59.778530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 215/436] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:21:59.803005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68bc64-dfb6-4a74-805f-0bcfad235256 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:59.838464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:21:59.839140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:21:59.839415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:21:59.839680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:21:59.840345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:21:59.840638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:21:59.842020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:21:59.852916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 222/437] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:21:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:21:59.858801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9592f-ef51-4c73-b33d-ba52140ce158 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:21:59.964697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9592f-ef51-4c73-b33d-ba52140ce158 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:21:59.994792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:21:59.999959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a6398-4577-43a7-b68d-681e1247a234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.000630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6398-4577-43a7-b68d-681e1247a234 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.000907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6398-4577-43a7-b68d-681e1247a234 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.001175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a6398-4577-43a7-b68d-681e1247a234 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.065150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.070964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:00.080581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a6398-4577-43a7-b68d-681e1247a234 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:00.082153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a6398-4577-43a7-b68d-681e1247a234 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:00.208850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad570d9-b692-429d-b480-71bc094369bc None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:00.211851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 216/438] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:00.219577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a6398-4577-43a7-b68d-681e1247a234 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.254559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607aa1f-a519-49df-8bc0-cce530588f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.255239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607aa1f-a519-49df-8bc0-cce530588f6c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.255522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607aa1f-a519-49df-8bc0-cce530588f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.255790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607aa1f-a519-49df-8bc0-cce530588f6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.261001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:00.268438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607aa1f-a519-49df-8bc0-cce530588f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4607aa1f-a519-49df-8bc0-cce530588f6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:00.273683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:00.274748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 223/439] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:00.278921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f4aa2-a25e-485b-b449-bce6db23bdb4 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.281503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f4aa2-a25e-485b-b449-bce6db23bdb4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.281503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f4aa2-a25e-485b-b449-bce6db23bdb4 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.281503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 224/440] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:00.282765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f4aa2-a25e-485b-b449-bce6db23bdb4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.334523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a6398-4577-43a7-b68d-681e1247a234 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:00.347804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:00.369424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd26c2-3e06-4e61-8cb6-6f6e614df675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.370056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd26c2-3e06-4e61-8cb6-6f6e614df675 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.370330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd26c2-3e06-4e61-8cb6-6f6e614df675 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.370622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd26c2-3e06-4e61-8cb6-6f6e614df675 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.371796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd26c2-3e06-4e61-8cb6-6f6e614df675 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:00.373248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd26c2-3e06-4e61-8cb6-6f6e614df675 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:00.374885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 217/441] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:00.375968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb42275-acd0-42ed-a277-bf10dc9c0084 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.376296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb42275-acd0-42ed-a277-bf10dc9c0084 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.376506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb42275-acd0-42ed-a277-bf10dc9c0084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.388507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f4aa2-a25e-485b-b449-bce6db23bdb4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:00.396189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fdb42275-acd0-42ed-a277-bf10dc9c0084 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:00.397710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fdb42275-acd0-42ed-a277-bf10dc9c0084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:00.422683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.423320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.423600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.423876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.424547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:00.424852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:00.426243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:00.435150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 225/442] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:00.435939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.467090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 218/443] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:00.467927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb42275-acd0-42ed-a277-bf10dc9c0084 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.499472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:00.549601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301e9cd-17e1-477a-9c47-17a74877188d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:00.575861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb42275-acd0-42ed-a277-bf10dc9c0084 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:00.577974 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:00.587593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16831b-b358-48e6-8ac8-288169621118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.588224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16831b-b358-48e6-8ac8-288169621118 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.588497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16831b-b358-48e6-8ac8-288169621118 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.588769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16831b-b358-48e6-8ac8-288169621118 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.589890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16831b-b358-48e6-8ac8-288169621118 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:00.591399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16831b-b358-48e6-8ac8-288169621118 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:00.592993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 226/444] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:00.598588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16831b-b358-48e6-8ac8-288169621118 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.621733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6d56-5808-4ddc-b2d7-a0f36e7f21c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.622835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6d56-5808-4ddc-b2d7-a0f36e7f21c1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.623321 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6d56-5808-4ddc-b2d7-a0f36e7f21c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.623797 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6d56-5808-4ddc-b2d7-a0f36e7f21c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.702257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d6d56-5808-4ddc-b2d7-a0f36e7f21c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:00.703750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d6d56-5808-4ddc-b2d7-a0f36e7f21c1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:00.721302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16831b-b358-48e6-8ac8-288169621118 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:00.734458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:00.756371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.756979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.757258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.757530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.758263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] RBAC: Adding query filter params (group.id=9b637c99470c4db3a99e64dfef03e668, role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=a33c93942a544564b55765288214ed83) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:00.758568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:00.760102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:00.765839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 227/445] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/role_assignments?group.id=9b637c99470c4db3a99e64dfef03e668&scope.project.id=a33c93942a544564b55765288214ed83&role.id=d06115dbab78488182bcb432bf9bec11 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:00.766607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.813840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 219/446] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:00.820037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6d56-5808-4ddc-b2d7-a0f36e7f21c1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.846150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:00.893801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446ceca-7e82-450d-b88b-703975f8385d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:00.932538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea563f8-403a-4bb2-8cd7-0042318be6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.932538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea563f8-403a-4bb2-8cd7-0042318be6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.932808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea563f8-403a-4bb2-8cd7-0042318be6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.933044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea563f8-403a-4bb2-8cd7-0042318be6b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.933135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6d56-5808-4ddc-b2d7-a0f36e7f21c1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:00.945974 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea563f8-403a-4bb2-8cd7-0042318be6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7ea563f8-403a-4bb2-8cd7-0042318be6b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:00.953094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:00.953997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:00.954866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 228/447] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:00.960927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea563f8-403a-4bb2-8cd7-0042318be6b3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:00.970062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:00.977128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375b02b-3416-4124-87cf-2a42554bf943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:00.977916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375b02b-3416-4124-87cf-2a42554bf943 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:00.978023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375b02b-3416-4124-87cf-2a42554bf943 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:00.978289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375b02b-3416-4124-87cf-2a42554bf943 None admin] PATH_INFO: `/v3/projects/a33c93942a544564b55765288214ed83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:00.979619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375b02b-3416-4124-87cf-2a42554bf943 None admin] RBAC: Authorizing `identity:get_project(project_id=a33c93942a544564b55765288214ed83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:00.981062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375b02b-3416-4124-87cf-2a42554bf943 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:00.982706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 220/448] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/projects/a33c93942a544564b55765288214ed83 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:00.988545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375b02b-3416-4124-87cf-2a42554bf943 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:01.070664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea563f8-403a-4bb2-8cd7-0042318be6b3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:01.087575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:01.091321 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375b02b-3416-4124-87cf-2a42554bf943 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:01.104767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:01.105316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:01.105591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:01.105865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:01.106542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:01.106881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:01.108277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:01.117525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 229/449] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:01.123150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:01.124763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df792014-6a26-4459-abf9-9d20e95881b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:01.125395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df792014-6a26-4459-abf9-9d20e95881b6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:01.125675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df792014-6a26-4459-abf9-9d20e95881b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:01.125943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df792014-6a26-4459-abf9-9d20e95881b6 None admin] PATH_INFO: `/v3/groups/9b637c99470c4db3a99e64dfef03e668` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:01.127093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df792014-6a26-4459-abf9-9d20e95881b6 None admin] RBAC: Authorizing `identity:get_group(group_id=9b637c99470c4db3a99e64dfef03e668)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:01.128531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df792014-6a26-4459-abf9-9d20e95881b6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:01.135163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 221/450] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:22:00 2022] GET /identity/v3/groups/9b637c99470c4db3a99e64dfef03e668 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:01.135823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df792014-6a26-4459-abf9-9d20e95881b6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:01.198445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:01.221558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aebee-1b77-4e01-a5e6-b6fb247d097f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:01.233880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df792014-6a26-4459-abf9-9d20e95881b6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:01.255018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abcd01-dd4d-483b-b512-30b99511d85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:01.255619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abcd01-dd4d-483b-b512-30b99511d85c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:01.255892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abcd01-dd4d-483b-b512-30b99511d85c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:01.256159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abcd01-dd4d-483b-b512-30b99511d85c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:01.269211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:01.269808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:01.270082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:01.270356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:01.271111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] RBAC: Adding query filter params (group.id=9b637c99470c4db3a99e64dfef03e668, role.id=5bbcda16701c451f9419aefc4671840e, scope.project.id=a33c93942a544564b55765288214ed83) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:01.271456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:01.272888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:01.279198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 222/451] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:22:01 2022] GET /identity/v3/role_assignments?group.id=9b637c99470c4db3a99e64dfef03e668&scope.project.id=a33c93942a544564b55765288214ed83&role.id=5bbcda16701c451f9419aefc4671840e => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:01.310998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:01.341693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abcd01-dd4d-483b-b512-30b99511d85c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:01.343205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abcd01-dd4d-483b-b512-30b99511d85c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:01.449403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 230/452] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:01.455298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:01.559927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d5bc7-4db6-4b72-b659-1e4a1ef4d9e4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:01.588981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:01.595510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6767f49-f664-4c29-ab29-66775156d64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:01.596116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6767f49-f664-4c29-ab29-66775156d64b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:01.596401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6767f49-f664-4c29-ab29-66775156d64b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:01.596667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6767f49-f664-4c29-ab29-66775156d64b None admin] PATH_INFO: `/v3/projects/529acf6d876d441682b1d4508345bc61` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:01.597849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6767f49-f664-4c29-ab29-66775156d64b None admin] RBAC: Authorizing `identity:get_project(project_id=529acf6d876d441682b1d4508345bc61)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:01.599321 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6767f49-f664-4c29-ab29-66775156d64b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:01.600942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 223/453] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:22:01 2022] GET /identity/v3/projects/529acf6d876d441682b1d4508345bc61 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:01.607173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abcd01-dd4d-483b-b512-30b99511d85c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:01.689951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:01.709756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abcd01-dd4d-483b-b512-30b99511d85c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:01.744040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56049d-0c10-4287-a98b-c13d3daae6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:01.745298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56049d-0c10-4287-a98b-c13d3daae6cf None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:01.745298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56049d-0c10-4287-a98b-c13d3daae6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:01.745479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56049d-0c10-4287-a98b-c13d3daae6cf None admin] PATH_INFO: `/v3/groups/b5ebac73de0c451c914c4f7224d20f23` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:01.746589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56049d-0c10-4287-a98b-c13d3daae6cf None admin] RBAC: Authorizing `identity:get_group(group_id=b5ebac73de0c451c914c4f7224d20f23)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:01.748075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56049d-0c10-4287-a98b-c13d3daae6cf None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:01.749649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 231/454] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Aug 31 04:22:01 2022] GET /identity/v3/groups/b5ebac73de0c451c914c4f7224d20f23 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:01.755837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6767f49-f664-4c29-ab29-66775156d64b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:01.859351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6767f49-f664-4c29-ab29-66775156d64b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:01.893745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61467270-4817-49ed-9f38-578670f74257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:01.894331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61467270-4817-49ed-9f38-578670f74257 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:01.894601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61467270-4817-49ed-9f38-578670f74257 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:01.894974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61467270-4817-49ed-9f38-578670f74257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:01.895697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61467270-4817-49ed-9f38-578670f74257 None admin] RBAC: Adding query filter params (group.id=b5ebac73de0c451c914c4f7224d20f23, role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=529acf6d876d441682b1d4508345bc61) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:01.895995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61467270-4817-49ed-9f38-578670f74257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:01.897412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61467270-4817-49ed-9f38-578670f74257 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:01.903084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 224/455] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Aug 31 04:22:01 2022] GET /identity/v3/role_assignments?group.id=b5ebac73de0c451c914c4f7224d20f23&scope.project.id=529acf6d876d441682b1d4508345bc61&role.id=241e99d365564c0fa95cb712f4004785 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:04.101509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16b575-b649-4189-88bd-4f2911f66b34 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:04.102035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16b575-b649-4189-88bd-4f2911f66b34 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:04.102035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16b575-b649-4189-88bd-4f2911f66b34 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:04.103774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 232/456] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:04.108789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3df0d-a3d1-4329-8a93-291b02cd0ac8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:04.109600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3df0d-a3d1-4329-8a93-291b02cd0ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:04.110085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3df0d-a3d1-4329-8a93-291b02cd0ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:04.132941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7dd3df0d-a3d1-4329-8a93-291b02cd0ac8 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:04.134450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7dd3df0d-a3d1-4329-8a93-291b02cd0ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:04.210386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:04.225609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 225/457] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:04.395122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0574-f28e-4f9d-95cd-f50a73062477 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:04.395294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0574-f28e-4f9d-95cd-f50a73062477 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:04.395576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0574-f28e-4f9d-95cd-f50a73062477 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:04.397127 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 233/458] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:04.402513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f9144-1cc5-4474-94ae-805c0ec148fd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:04.402829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f9144-1cc5-4474-94ae-805c0ec148fd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:04.403101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f9144-1cc5-4474-94ae-805c0ec148fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:04.427708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-796f9144-1cc5-4474-94ae-805c0ec148fd None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:04.428895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-796f9144-1cc5-4474-94ae-805c0ec148fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:04.521316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 226/459] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:22:04.596488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da0574-f28e-4f9d-95cd-f50a73062477 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:04.711154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc902-ab21-48a4-81a2-01065dae1c06 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:04.711335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc902-ab21-48a4-81a2-01065dae1c06 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:04.711606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc902-ab21-48a4-81a2-01065dae1c06 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:04.713341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 227/460] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:04.718540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad97a-02c5-4798-a691-f1d204629f47 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:04.718946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad97a-02c5-4798-a691-f1d204629f47 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:04.719222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad97a-02c5-4798-a691-f1d204629f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:04.727238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da0574-f28e-4f9d-95cd-f50a73062477 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:04.755398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-346ad97a-02c5-4798-a691-f1d204629f47 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:04.756869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-346ad97a-02c5-4798-a691-f1d204629f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:04.807558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:04.808209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:04.808482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:04.808751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:04.832661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:04.834171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:04.834277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 228/461] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:04.850025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:04.850969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:04.851780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:04.851780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:04.851780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:04.851780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 234/462] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:04.852514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:04.855877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ad97a-02c5-4798-a691-f1d204629f47 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:04.860624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:04.916761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:04.971907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ad97a-02c5-4798-a691-f1d204629f47 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.016706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ffcca-0654-4f03-a05f-c3736f525b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.017594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ffcca-0654-4f03-a05f-c3736f525b8b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.018133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ffcca-0654-4f03-a05f-c3736f525b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.018541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ffcca-0654-4f03-a05f-c3736f525b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.032192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ffcca-0654-4f03-a05f-c3736f525b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.033794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22350f16-66a6-479b-9647-fd76de5b1eb8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.034050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ffcca-0654-4f03-a05f-c3736f525b8b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-dc5ffcca-0654-4f03-a05f-c3736f525b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:05.053928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:05.055120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:05.056226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:05.056226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:05.056226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:05.059007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 229/463] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:05.060564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a82c6-715f-44d4-b198-19f99185d395 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.061073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a82c6-715f-44d4-b198-19f99185d395 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.061496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a82c6-715f-44d4-b198-19f99185d395 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.063004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 230/464] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:05.070292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2c4a0-d7b2-4468-9df4-6ac19ed38f62 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.070573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2c4a0-d7b2-4468-9df4-6ac19ed38f62 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.070876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2c4a0-d7b2-4468-9df4-6ac19ed38f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.087209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.088226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.088730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.089200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.090047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:05.090540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.092161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.093939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fac2c4a0-d7b2-4468-9df4-6ac19ed38f62 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:05.095398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fac2c4a0-d7b2-4468-9df4-6ac19ed38f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:05.116077 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 235/465] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:05.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.170750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 231/466] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:05.170750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2c4a0-d7b2-4468-9df4-6ac19ed38f62 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.184033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:05.215086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:05.232363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d88bd-e88e-46b6-929d-64ad21af2d55 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.270341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.271319 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.271787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.272240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.273144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:05.273674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.275284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.288048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 236/467] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:05.289107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.292749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2c4a0-d7b2-4468-9df4-6ac19ed38f62 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.328562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8326f5-d368-477a-b7cb-97b17ded9eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.329188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8326f5-d368-477a-b7cb-97b17ded9eb8 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.329460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8326f5-d368-477a-b7cb-97b17ded9eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.329725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8326f5-d368-477a-b7cb-97b17ded9eb8 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.330389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8326f5-d368-477a-b7cb-97b17ded9eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.331894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8326f5-d368-477a-b7cb-97b17ded9eb8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.362776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 232/468] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:22:05 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:05.362776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8326f5-d368-477a-b7cb-97b17ded9eb8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.442102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eaa60-b612-4381-a7c6-1877b7a5526d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.475198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:05.478311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:05.489380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8326f5-d368-477a-b7cb-97b17ded9eb8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.498092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.498924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.499209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.499485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.516148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.517714 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.534963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adf692-759d-4bbe-9fda-aa8562ebd241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.535628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adf692-759d-4bbe-9fda-aa8562ebd241 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.535908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adf692-759d-4bbe-9fda-aa8562ebd241 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.536179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adf692-759d-4bbe-9fda-aa8562ebd241 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.536845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10adf692-759d-4bbe-9fda-aa8562ebd241 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:05.536939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:05.537726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:05.538550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:05.538550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:05.538550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:05.538550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 237/469] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:05.538807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10adf692-759d-4bbe-9fda-aa8562ebd241 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.538875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.568598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 233/470] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:22:05 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:05.569409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adf692-759d-4bbe-9fda-aa8562ebd241 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.675372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddfb60-dc5d-4843-a1b0-b6c6dcab32ae None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.708884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adf692-759d-4bbe-9fda-aa8562ebd241 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.720747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.721396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.721673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.721946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.738369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.740926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:05.758206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:05.759398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:05.760463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:05.760463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:05.760463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:05.762307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 238/471] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:05.766826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.767866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.768994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.769690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.769964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.770661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:05.770982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.772407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.790707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 234/472] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:05.803096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.818990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:05.823461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:05.929592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4221e-4dfd-42fc-9bae-b38185272f45 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.954512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a450c-baba-4645-a788-125bc9bc9839 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.972847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3993c68-7933-4dde-8125-fb49d4ff739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.973496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993c68-7933-4dde-8125-fb49d4ff739e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.973778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993c68-7933-4dde-8125-fb49d4ff739e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.974051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993c68-7933-4dde-8125-fb49d4ff739e None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.974706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3993c68-7933-4dde-8125-fb49d4ff739e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:05.975043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3993c68-7933-4dde-8125-fb49d4ff739e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.976454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3993c68-7933-4dde-8125-fb49d4ff739e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.991295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 239/473] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:05.992310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453308e-e9e4-4562-bf4c-7a51d62f1241 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.992596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453308e-e9e4-4562-bf4c-7a51d62f1241 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.992873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453308e-e9e4-4562-bf4c-7a51d62f1241 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.994271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 240/474] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:05.998996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453308e-e9e4-4562-bf4c-7a51d62f1241 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:06.006055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a1a71-b6e6-4fa2-bad3-31611aaa2730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:06.006676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a1a71-b6e6-4fa2-bad3-31611aaa2730 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:06.006991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a1a71-b6e6-4fa2-bad3-31611aaa2730 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:06.007260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a1a71-b6e6-4fa2-bad3-31611aaa2730 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:06.008338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a1a71-b6e6-4fa2-bad3-31611aaa2730 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:06.009800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a1a71-b6e6-4fa2-bad3-31611aaa2730 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:06.034953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 235/475] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:22:05 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:06.036181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ae39c-df14-4663-93e9-ee8fdaee04ae None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:06.036600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ae39c-df14-4663-93e9-ee8fdaee04ae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:06.036879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ae39c-df14-4663-93e9-ee8fdaee04ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:06.056628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-335ae39c-df14-4663-93e9-ee8fdaee04ae None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:06.058296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-335ae39c-df14-4663-93e9-ee8fdaee04ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:06.110252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:06.128547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:06.133459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453308e-e9e4-4562-bf4c-7a51d62f1241 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:06.142028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 236/476] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:06.176559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aa19f-9532-4b8a-b58a-ae087b7077f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:06.177550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa19f-9532-4b8a-b58a-ae087b7077f3 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:06.178041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa19f-9532-4b8a-b58a-ae087b7077f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:06.178505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa19f-9532-4b8a-b58a-ae087b7077f3 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:06.179486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aa19f-9532-4b8a-b58a-ae087b7077f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:06.181176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aa19f-9532-4b8a-b58a-ae087b7077f3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:06.214512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 241/477] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:22:05 2022] POST /identity/v3/users => generated 249 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:06.461397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ae39c-df14-4663-93e9-ee8fdaee04ae None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:06.568657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ae39c-df14-4663-93e9-ee8fdaee04ae None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:06.610809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:06.611489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:06.611776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:06.612043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:06.612722 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:06.614108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:06.629659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.sql.core [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 04:22:06.629659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 04:22:06.629659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [parameters: {'id': 'f8dd4b0d563a41b7bf21bc64fd0493db', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 04:22:06.629659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81570) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Aug 31 04:22:06.655088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 04:22:06.656030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application yield resource Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise self.value Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application session.commit() Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 04:22:06.656917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.flush() Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application compat.raise_( Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 04:22:06.657739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 04:22:06.658611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 04:22:06.659438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [parameters: {'id': 'f8dd4b0d563a41b7bf21bc64fd0493db', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:06.660192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 04:22:06.660970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:06.663342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 237/478] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 31 04:22:06 2022] POST /identity/v3/roles => generated 144 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:06.663585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aa19f-9532-4b8a-b58a-ae087b7077f3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:06.800030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aa19f-9532-4b8a-b58a-ae087b7077f3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:06.838947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d93db-7828-49a1-87e4-f56ca6d0ccc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:06.839677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d93db-7828-49a1-87e4-f56ca6d0ccc6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:06.839954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d93db-7828-49a1-87e4-f56ca6d0ccc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:06.840217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d93db-7828-49a1-87e4-f56ca6d0ccc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:06.852909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:06.856179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d93db-7828-49a1-87e4-f56ca6d0ccc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-489d93db-7828-49a1-87e4-f56ca6d0ccc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:06.861852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:06.862700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:06.863509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 242/479] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:06.870926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:06.967748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:07.028062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3cc5b-1dd8-4cfe-b540-ea382ecf05d2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:07.074751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ac4f6-e53e-4def-a203-a2d17779824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:07.074751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac4f6-e53e-4def-a203-a2d17779824d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:07.074751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac4f6-e53e-4def-a203-a2d17779824d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:07.074751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac4f6-e53e-4def-a203-a2d17779824d None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:07.075096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ac4f6-e53e-4def-a203-a2d17779824d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:07.075096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ac4f6-e53e-4def-a203-a2d17779824d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:07.080131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ac4f6-e53e-4def-a203-a2d17779824d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:07.089304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 238/480] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:07.903974 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c57790-dcc1-40cd-92e7-40ae719ef3fc None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:07.904529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c57790-dcc1-40cd-92e7-40ae719ef3fc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:07.904529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c57790-dcc1-40cd-92e7-40ae719ef3fc None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:07.906129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 243/481] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:07.911374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1725b13-d375-44fd-bbbd-933fa9134886 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:07.911773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1725b13-d375-44fd-bbbd-933fa9134886 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:07.911971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1725b13-d375-44fd-bbbd-933fa9134886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:07.939084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b1725b13-d375-44fd-bbbd-933fa9134886 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:07.940280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b1725b13-d375-44fd-bbbd-933fa9134886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:08.036787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 239/482] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:08.172689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0f7d1-81eb-420b-928f-0604b0dd0544 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:08.172880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0f7d1-81eb-420b-928f-0604b0dd0544 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:08.173157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0f7d1-81eb-420b-928f-0604b0dd0544 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:08.174516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 244/483] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:08.179954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239040a-7f11-4652-ac42-9daad704f95e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:08.180306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239040a-7f11-4652-ac42-9daad704f95e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:08.180587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239040a-7f11-4652-ac42-9daad704f95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:08.207765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f239040a-7f11-4652-ac42-9daad704f95e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:08.209026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f239040a-7f11-4652-ac42-9daad704f95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:08.262285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:08.295842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 240/484] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:08.479672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0f7d1-81eb-420b-928f-0604b0dd0544 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:08.496016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db889ee4-4a36-493c-8582-a49dcdd1b6cb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:08.496326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db889ee4-4a36-493c-8582-a49dcdd1b6cb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:08.496575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db889ee4-4a36-493c-8582-a49dcdd1b6cb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:08.498209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 241/485] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:08.503230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce085b57-c9c4-4e3b-8027-e38bbac3b20e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:08.503726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce085b57-c9c4-4e3b-8027-e38bbac3b20e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:08.504164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce085b57-c9c4-4e3b-8027-e38bbac3b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:08.543727 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ce085b57-c9c4-4e3b-8027-e38bbac3b20e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:08.545206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce085b57-c9c4-4e3b-8027-e38bbac3b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:08.605866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0f7d1-81eb-420b-928f-0604b0dd0544 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:08.631181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 242/486] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:08.632654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aee005-73f5-4f76-8178-b79441e39cf8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:08.632946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aee005-73f5-4f76-8178-b79441e39cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:08.633219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aee005-73f5-4f76-8178-b79441e39cf8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:08.634650 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 243/487] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:08.644467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab84b4-8316-406a-bae0-13ba9acba1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:08.645345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab84b4-8316-406a-bae0-13ba9acba1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:08.645789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab84b4-8316-406a-bae0-13ba9acba1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:08.646222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab84b4-8316-406a-bae0-13ba9acba1a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:08.648469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac1da7-274d-48c6-ab3a-8f21cfbd1724 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:08.648755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac1da7-274d-48c6-ab3a-8f21cfbd1724 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:08.649033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac1da7-274d-48c6-ab3a-8f21cfbd1724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:08.658389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab84b4-8316-406a-bae0-13ba9acba1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b6ab84b4-8316-406a-bae0-13ba9acba1a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:08.664572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:08.665439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:08.666222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:08.669071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 245/488] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:08.676154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab84b4-8316-406a-bae0-13ba9acba1a0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:08.681995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-22ac1da7-274d-48c6-ab3a-8f21cfbd1724 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:08.683312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-22ac1da7-274d-48c6-ab3a-8f21cfbd1724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:08.762942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 244/489] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:22:08.763320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:08.764703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac1da7-274d-48c6-ab3a-8f21cfbd1724 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:08.819341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab84b4-8316-406a-bae0-13ba9acba1a0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:08.844390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:08.897861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:08.899757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:08.900492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:08.902581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:08.903478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:08.903922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:08.906860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:08.930920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 246/490] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:08.938381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:08.960183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac1da7-274d-48c6-ab3a-8f21cfbd1724 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.006345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5355b6-5aa5-4264-a4fe-28724717dfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.007008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5355b6-5aa5-4264-a4fe-28724717dfc1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.007335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5355b6-5aa5-4264-a4fe-28724717dfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.007609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5355b6-5aa5-4264-a4fe-28724717dfc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.018326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5355b6-5aa5-4264-a4fe-28724717dfc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7c5355b6-5aa5-4264-a4fe-28724717dfc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:09.023204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:09.024100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:09.025297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 245/491] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:09.026073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5355b6-5aa5-4264-a4fe-28724717dfc1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.061285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e96455-e354-41da-9086-50fcbc0f01fe None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.093398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:09.108752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.109390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.109674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.109944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.121666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.123173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.132519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:09.136639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5355b6-5aa5-4264-a4fe-28724717dfc1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:09.139260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:09.140100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:09.140915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:09.140915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:09.140915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:09.140915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 247/492] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:09.141508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.180825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7d2a-3757-4e11-be3b-507b4b6c9cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.181455 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7d2a-3757-4e11-be3b-507b4b6c9cdb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.181735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7d2a-3757-4e11-be3b-507b4b6c9cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.182187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7d2a-3757-4e11-be3b-507b4b6c9cdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.193576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee7d2a-3757-4e11-be3b-507b4b6c9cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5cee7d2a-3757-4e11-be3b-507b4b6c9cdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:09.198530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:09.199529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:09.200294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 246/493] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:09.200942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7d2a-3757-4e11-be3b-507b4b6c9cdb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.252362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45da2d-c179-4d71-b732-819a351d234d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.287005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.287527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.287806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.288072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.288739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:09.289039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.290410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.298661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 248/494] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:09.300857 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.312843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7d2a-3757-4e11-be3b-507b4b6c9cdb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.339523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:09.354654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2bd37-9e10-4cf4-a5cd-c6ad5d530980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.355318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2bd37-9e10-4cf4-a5cd-c6ad5d530980 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.355593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2bd37-9e10-4cf4-a5cd-c6ad5d530980 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.355856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2bd37-9e10-4cf4-a5cd-c6ad5d530980 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.366852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2bd37-9e10-4cf4-a5cd-c6ad5d530980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-efa2bd37-9e10-4cf4-a5cd-c6ad5d530980 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:09.371843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:09.372700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:09.373479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:09.378006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 247/495] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:09.378666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2bd37-9e10-4cf4-a5cd-c6ad5d530980 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.398757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:09.412606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b169852-6d5f-4aa7-9cf6-dfba46220fc0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.451862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.452635 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.452920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.453969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.454641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:09.454975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.456467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.468675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 249/496] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:09.469491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.491780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2bd37-9e10-4cf4-a5cd-c6ad5d530980 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.528268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.528871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.529168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.529419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.530093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:09.530363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.531811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.540357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 248/497] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:09.543844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.570085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:09.577355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ceab77-26db-4abf-8d19-d2a51d03ef3a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.613018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.613657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.613931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.614234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.625014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.626486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.630508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:09.643019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:09.643940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:09.644742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:09.644742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:09.644742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:09.644742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 250/498] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:09.644962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.657065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72225f-3677-468d-b40c-bdf21c0facd1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.692373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.692976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.693250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.693516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.694182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:09.694482 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.695879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.703910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 249/499] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:09.704699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.748894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea816b3-adea-4a12-b214-ab063f9d992a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.781542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.782161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.782530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.782826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.791122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.792522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.803961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd716ac7-b487-48be-ab3b-5637e4602a0a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:09.804703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:09.805543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:09.806455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 251/500] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:09.807178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.820161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:09.847741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652b03-6208-4c04-a268-c34843093987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.848372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652b03-6208-4c04-a268-c34843093987 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.848653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652b03-6208-4c04-a268-c34843093987 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.848930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652b03-6208-4c04-a268-c34843093987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.861026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85652b03-6208-4c04-a268-c34843093987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.862557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85652b03-6208-4c04-a268-c34843093987 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.864405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-85652b03-6208-4c04-a268-c34843093987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:09.878065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:09.879022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:09.879852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:09.879852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:09.879852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:09.879852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 250/501] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:09.880518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669aba2-1ae2-44cb-b09e-04b1975b5e98 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.880697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669aba2-1ae2-44cb-b09e-04b1975b5e98 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.880852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669aba2-1ae2-44cb-b09e-04b1975b5e98 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.882265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 251/502] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:09.882769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669aba2-1ae2-44cb-b09e-04b1975b5e98 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.911848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0d63a-e503-4e20-8eff-55d1195be91d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:09.945663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:09.946262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:09.946570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:09.946869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:09.947516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:09.947957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:09.949475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:09.961104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 252/503] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:09.961890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:09.989273 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669aba2-1ae2-44cb-b09e-04b1975b5e98 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.023726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.024343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.024612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.024878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.027828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.035683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.037164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:10.051199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:10.052084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:10.052895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:10.052895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:10.052895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:10.052895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 252/504] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:10.053112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.067250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b071c-3464-4bce-8a20-7f5ded5b437a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.099781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.100721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead50e19-b3bf-4bbd-b2da-c4b4361254df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.101330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead50e19-b3bf-4bbd-b2da-c4b4361254df None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.101602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead50e19-b3bf-4bbd-b2da-c4b4361254df None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.101867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead50e19-b3bf-4bbd-b2da-c4b4361254df None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.102696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead50e19-b3bf-4bbd-b2da-c4b4361254df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:10.103022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead50e19-b3bf-4bbd-b2da-c4b4361254df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.104540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead50e19-b3bf-4bbd-b2da-c4b4361254df None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.145206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 253/505] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:22:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:10.145206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd19f3-e593-4df7-a680-6167b647aa36 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.145206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd19f3-e593-4df7-a680-6167b647aa36 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.145206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd19f3-e593-4df7-a680-6167b647aa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.155611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8683de5-72cc-4043-a3d2-fa797e687625 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.160146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c6cd19f3-e593-4df7-a680-6167b647aa36 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:10.161326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c6cd19f3-e593-4df7-a680-6167b647aa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:10.189997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.190602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.190914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.191208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.192228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:10.192524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.193895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.206251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 253/506] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:10.207084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.232431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 254/507] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:10.233275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd19f3-e593-4df7-a680-6167b647aa36 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.294983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.310321 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150610c-b1a5-4f5e-ab73-234a98e68332 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.318470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.347197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd19f3-e593-4df7-a680-6167b647aa36 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.352434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.353069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.353346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.353617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.361369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.362908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:10.376306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:10.377253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:10.378142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 254/508] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:10.378675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.388185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.388805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.389083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.389343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.389999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:10.390296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.391713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.403536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 255/509] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:10.404177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.485723 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa25de-1e16-4f72-aed4-7c9e7cbb7c68 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.508324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1817-3252-46bb-9274-963d598c1b21 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.519374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08671b36-9667-43a9-af13-a82c9c98d61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.519981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08671b36-9667-43a9-af13-a82c9c98d61d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.520255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08671b36-9667-43a9-af13-a82c9c98d61d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.520518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08671b36-9667-43a9-af13-a82c9c98d61d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.524127 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.541583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.542205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.542478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.542768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.551067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.552532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.553726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:10.564005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:10.565063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 256/510] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:10.565922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.588751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08671b36-9667-43a9-af13-a82c9c98d61d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.590164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08671b36-9667-43a9-af13-a82c9c98d61d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.669116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be99b2-3b33-4a8f-b28d-172f66cbfc75 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.688104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 255/511] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:10.689001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08671b36-9667-43a9-af13-a82c9c98d61d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.705629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.706331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.706608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.706906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.707649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:10.707953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.709399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.726330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 257/512] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:10.727376 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.753636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.761126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.787632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08671b36-9667-43a9-af13-a82c9c98d61d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.822949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.823509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.823785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.824047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.828940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efe230-883d-4b28-8b13-8089d2d6fcd9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.831254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.832718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:10.845003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:10.846037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 256/513] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:10.846946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.866019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.866652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.866967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.867233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.877341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.878926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:10.894302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:10.895368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:10.896295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:10.896295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:10.896295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:10.896295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 258/514] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:10.896587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:10.946645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289d021-0ccf-426a-86b2-4e4a48e9c27a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:10.971922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.981166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:10.981761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:10.981916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:10.982355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:10.982355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:10.983057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:10.983332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:10.984802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:10.996930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fd6b6-e7c0-4b46-877a-9b00abd9f388 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.021226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 257/515] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:11.021868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.034012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.034653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.034958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.035228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.045913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.047367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:11.061202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:11.062218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:11.063082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:11.063082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:11.063082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:11.063082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 259/516] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:11.063728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.131329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac648d37-5036-49b6-87ba-b1090259b98b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.166175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57b59a-3281-4101-8854-906cb53dd143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.166864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57b59a-3281-4101-8854-906cb53dd143 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.167147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57b59a-3281-4101-8854-906cb53dd143 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.167433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57b59a-3281-4101-8854-906cb53dd143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.171824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53389d0c-da89-43b0-9002-c8a17b73942b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.206592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.207243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.207526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.207798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.208532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:11.208836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.210285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.218611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:11.219060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:11.229323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 260/517] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:11.230075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.237240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57b59a-3281-4101-8854-906cb53dd143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.239146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57b59a-3281-4101-8854-906cb53dd143 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.333078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa555c-f255-48d2-8334-fe3c83e42557 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.337279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 258/518] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:11.338112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57b59a-3281-4101-8854-906cb53dd143 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.368310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.368920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.369196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.369461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.370213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:11.370508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.371921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.382925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 261/519] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:11.383741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.420432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:11.436550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57b59a-3281-4101-8854-906cb53dd143 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.440284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:11.469742 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0df71-db2c-46be-8cd0-9b1ba65273f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.470344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0df71-db2c-46be-8cd0-9b1ba65273f1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.470615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0df71-db2c-46be-8cd0-9b1ba65273f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.470919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0df71-db2c-46be-8cd0-9b1ba65273f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.484183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a15dc-d06e-4dfd-b2eb-5239883e546e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.517124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.517731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.518006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.518282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.518963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:11.519264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.520644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.532197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 262/520] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:11.532988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.539905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0df71-db2c-46be-8cd0-9b1ba65273f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.541361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0df71-db2c-46be-8cd0-9b1ba65273f1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.624980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:11.630374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8941df6-8711-48be-be75-56e70324b6ee None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.643605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 259/521] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:11.644168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0df71-db2c-46be-8cd0-9b1ba65273f1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.649771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:11.663944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1a956-c34b-4955-a515-8329cdec27f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.664561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1a956-c34b-4955-a515-8329cdec27f3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.664822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1a956-c34b-4955-a515-8329cdec27f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.665089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1a956-c34b-4955-a515-8329cdec27f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.736131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1a956-c34b-4955-a515-8329cdec27f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.737589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1a956-c34b-4955-a515-8329cdec27f3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.747334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0df71-db2c-46be-8cd0-9b1ba65273f1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.781886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.782506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.782803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.783070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.793323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.794799 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:11.811132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:11.812073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:11.812911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:11.812911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:11.812911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:11.812911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 260/522] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:11.813133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.841709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 263/523] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:11.842555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1a956-c34b-4955-a515-8329cdec27f3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.850106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:11.861451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:11.919037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893804a-991c-4192-af06-3adf13b3b2d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.944680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1a956-c34b-4955-a515-8329cdec27f3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:11.954662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5558c-1a75-43fd-ab9c-ab2fd64232a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.955296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5558c-1a75-43fd-ab9c-ab2fd64232a1 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.955548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5558c-1a75-43fd-ab9c-ab2fd64232a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.955829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5558c-1a75-43fd-ab9c-ab2fd64232a1 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.956469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5558c-1a75-43fd-ab9c-ab2fd64232a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.957917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5558c-1a75-43fd-ab9c-ab2fd64232a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:11.981522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:11.981721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 261/524] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:22:11 2022] POST /identity/v3/users => generated 247 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:11.982412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5558c-1a75-43fd-ab9c-ab2fd64232a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:11.982613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:11.983387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:11.983798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:11.991716 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:11.993220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:12.006447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:12.007521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:12.008345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 264/525] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:12.009096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.108703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:12.111538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:12.124612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5558c-1a75-43fd-ab9c-ab2fd64232a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.142196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9e234-1233-4100-a2e1-52a72321df44 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.168377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.169333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.169550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.170334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.181926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.183137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.183357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.183610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.183921 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.184260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:12.184558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.185415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.185920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.198774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 265/526] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:12.199707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:12.205010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:12.205911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:12.206793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:12.206793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:12.206793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:12.206793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 262/527] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:12.207399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.304420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122104c-dc90-4757-a4ec-570d79ca1a88 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.311513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ebfcb-be1e-432f-b7c4-84a51cf8f71c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.340498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.341146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.341477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.341762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.344984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:12.350222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.350829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.351110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.351380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.352076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:12.352378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.353835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.353931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.355509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:12.371560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:12.372710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:12.373521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:12.373521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:12.373521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:12.373521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 266/528] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:12.373521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.375693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 263/529] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:12.376676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.381676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:12.492991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dc0b1-09c5-4a64-afeb-c7fc193dc893 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.504528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc266d36-033e-4835-a57c-9ead7bb4c974 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.531542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.532223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.532593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.532848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.539580 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.539731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.540189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.540460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.540726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.541096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.541368 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:12.542065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.543448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:12.553865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:12.554668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:12.555775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:12.555775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:12.555775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:12.555775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:12.555775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:12.555775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:12.555775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:12.555775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:12.560000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 264/530] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:12.560185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 267/531] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:12.560863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.561045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.613448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:12.625122 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:12.663376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d58c8-55a2-4eec-948e-a06ead0275fc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.669789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ebf54-eb0e-4e8a-b53f-876f39f7d95d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.697056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.697658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.697979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.698242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.698996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:12.699289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.700678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.705468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.706296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.706898 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.707168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.707901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=708ce6cfc5f64d26a418b4339fd883fb) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:12.708205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.709758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.716294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 265/532] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:12.716294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.720641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 268/533] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/role_assignments?user.id=708ce6cfc5f64d26a418b4339fd883fb&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:12.726816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.833160 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f8d1-4105-4edb-b587-d9676782aaf0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.859956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:12.861892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d62273-3b6d-47ef-90c1-efc94382f751 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:12.873270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.874004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.874857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.874857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.882345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.884018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.885493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:12.899527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:12.900394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:12.901333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 266/534] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:12.901939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:12.918667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:12.919620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:12.920188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:12.920469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:12.921187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:12.921491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:12.923518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:12.950890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 269/535] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:12.951987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:13.021895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8f9f1-e1c5-416b-afb8-4df04b8f6210 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:13.057996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5f663-bf44-4891-8ffb-3edf99233e42 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:13.076727 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:13.077369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:13.077639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:13.077903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:13.087875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:13.091013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:13.100311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:13.100928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:13.101200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:13.101465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:13.102151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:13.102449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:13.103908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:13.108124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:13.108923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:13.109719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:13.109719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:13.109719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:13.109719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:13.109719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:13.109719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:13.109719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:13.109719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:13.112699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 267/536] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:13.114623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:13.122904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 270/537] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:13.122904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:13.158968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:13.183139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:13.234865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec861eb-2ba2-4110-a725-26abe367280c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:13.256123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc67e2-9e53-40e2-9046-874f5aacedf3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:13.274908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:13.275728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:13.276163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:13.276566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:13.277388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:13.277850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:13.279446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:13.296782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 271/538] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:13.298048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:13.299906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:13.300620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:13.301006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:13.301628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:13.303451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=ca15f81281dc42f4befeb9335698d18f) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:13.303876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:13.305413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:13.314792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 268/539] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:13 2022] GET /identity/v3/role_assignments?user.id=ca15f81281dc42f4befeb9335698d18f&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:13.315777 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:13.470256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:13.486452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd40d2f-2740-4a25-b840-8f7f5abc5bc0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:13.513665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5926934-80da-4f31-b45a-d89af867e1d6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:13.526363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:13.534252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e45b23-70dd-40af-b44c-648da0bb5a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:13.535237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e45b23-70dd-40af-b44c-648da0bb5a84 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:13.535516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e45b23-70dd-40af-b44c-648da0bb5a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:13.535789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e45b23-70dd-40af-b44c-648da0bb5a84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:13.536557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e45b23-70dd-40af-b44c-648da0bb5a84 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=6f3a86b76cff4b48a382888cbac913f4) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:13.536866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e45b23-70dd-40af-b44c-648da0bb5a84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:13.538330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e45b23-70dd-40af-b44c-648da0bb5a84 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:13.544521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 272/540] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:13 2022] GET /identity/v3/role_assignments?user.id=6f3a86b76cff4b48a382888cbac913f4&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:13.561715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb9fc0-c31d-48a2-b075-f27b9e3ef2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:13.562709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb9fc0-c31d-48a2-b075-f27b9e3ef2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:13.563275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb9fc0-c31d-48a2-b075-f27b9e3ef2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:13.563729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb9fc0-c31d-48a2-b075-f27b9e3ef2e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:13.564696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb9fc0-c31d-48a2-b075-f27b9e3ef2e2 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=13603c6cd9544f5391256ea600ad0709) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:13.565202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb9fc0-c31d-48a2-b075-f27b9e3ef2e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:13.567351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb9fc0-c31d-48a2-b075-f27b9e3ef2e2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:13.574462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 269/541] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:13 2022] GET /identity/v3/role_assignments?user.id=13603c6cd9544f5391256ea600ad0709&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:14.107117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32fc60-c44e-453c-ad88-2bebe64c9159 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:14.107687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32fc60-c44e-453c-ad88-2bebe64c9159 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:14.107687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32fc60-c44e-453c-ad88-2bebe64c9159 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:14.109311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 273/542] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:14.120502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada17de-e754-480f-a64e-e8096df159c3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:14.121493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada17de-e754-480f-a64e-e8096df159c3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:14.121953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada17de-e754-480f-a64e-e8096df159c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:14.151098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1ada17de-e754-480f-a64e-e8096df159c3 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:14.152724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1ada17de-e754-480f-a64e-e8096df159c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:14.249435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 270/543] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:22:14.568500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32fc60-c44e-453c-ad88-2bebe64c9159 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:14.671959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32fc60-c44e-453c-ad88-2bebe64c9159 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:14.720650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2612e-1675-4732-b129-909cc6a9dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:14.721695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2612e-1675-4732-b129-909cc6a9dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:14.722161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2612e-1675-4732-b129-909cc6a9dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:14.722617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2612e-1675-4732-b129-909cc6a9dbd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:14.743582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2612e-1675-4732-b129-909cc6a9dbd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9af2612e-1675-4732-b129-909cc6a9dbd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:14.748836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:14.749663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:14.750445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:14.752686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 274/544] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:14.758655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada17de-e754-480f-a64e-e8096df159c3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:14.845170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d10cb-b86b-42fa-b73a-05f0368e5e6c None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:14.845696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d10cb-b86b-42fa-b73a-05f0368e5e6c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:14.845973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d10cb-b86b-42fa-b73a-05f0368e5e6c None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:14.848212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 275/545] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:22:14.853147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e31e6-fc85-456c-99bb-ae11bd9c4ad6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:14.853449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e31e6-fc85-456c-99bb-ae11bd9c4ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:14.853731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e31e6-fc85-456c-99bb-ae11bd9c4ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:14.879165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-339e31e6-fc85-456c-99bb-ae11bd9c4ad6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:14.880334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-339e31e6-fc85-456c-99bb-ae11bd9c4ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:14.895080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:14.904807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada17de-e754-480f-a64e-e8096df159c3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:14.906653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:14.966931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:14.967549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:14.967823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:14.968089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:14.968758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:14.969048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:14.970429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:14.983221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 271/546] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:14.989205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:15.005792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 276/547] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:15.144082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf823a-ea5f-4467-835d-330417d22780 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:15.187575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:15.188219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:15.188499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:15.188774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:15.201708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:15.203232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:15.219140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:15.220077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:15.220914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:15.220914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:15.220914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:15.220914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 272/548] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:15.226202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e31e6-fc85-456c-99bb-ae11bd9c4ad6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:15.326651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:15.358409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:15.381840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e31e6-fc85-456c-99bb-ae11bd9c4ad6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:15.452552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:15.453091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:15.453364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:15.453636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:15.454286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:15.454580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:15.455979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:15.486108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 277/549] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:15.491984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:15.507089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:15.519067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621df13-f811-4c57-818f-0bbdc0a7cfa4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:15.564105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd5605-f8f7-4003-ab7e-0ccabb5da1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:15.564751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd5605-f8f7-4003-ab7e-0ccabb5da1ed None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:15.565022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd5605-f8f7-4003-ab7e-0ccabb5da1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:15.565383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd5605-f8f7-4003-ab7e-0ccabb5da1ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:15.599316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd5605-f8f7-4003-ab7e-0ccabb5da1ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e7bd5605-f8f7-4003-ab7e-0ccabb5da1ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:15.605040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:15.606487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:15.607547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:15.610015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 273/550] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:15.615336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd5605-f8f7-4003-ab7e-0ccabb5da1ed None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:15.629904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e91ee7-76c2-4aa4-b8a6-0dd8b4591477 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:15.690285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ae7f9-b229-4ec6-aee4-a6dd44c02cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:15.690963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ae7f9-b229-4ec6-aee4-a6dd44c02cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:15.691462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ae7f9-b229-4ec6-aee4-a6dd44c02cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:15.691755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ae7f9-b229-4ec6-aee4-a6dd44c02cb7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:15.702650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ae7f9-b229-4ec6-aee4-a6dd44c02cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:15.704742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ae7f9-b229-4ec6-aee4-a6dd44c02cb7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-326ae7f9-b229-4ec6-aee4-a6dd44c02cb7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:15.719066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:15.720029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:15.721114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:15.723666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 278/551] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:15.729495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e77a68-1983-4920-9cc1-496141b26a27 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:15.729629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:15.730290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e77a68-1983-4920-9cc1-496141b26a27 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:15.730797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e77a68-1983-4920-9cc1-496141b26a27 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:15.732683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 279/552] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:15.733673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e77a68-1983-4920-9cc1-496141b26a27 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:15.757750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd5605-f8f7-4003-ab7e-0ccabb5da1ed None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:15.794553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf231fc-7d04-429b-be4f-d6d48151b954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:15.795218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf231fc-7d04-429b-be4f-d6d48151b954 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:15.795488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf231fc-7d04-429b-be4f-d6d48151b954 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:15.795754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf231fc-7d04-429b-be4f-d6d48151b954 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:15.796433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf231fc-7d04-429b-be4f-d6d48151b954 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:15.796726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf231fc-7d04-429b-be4f-d6d48151b954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:15.798231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf231fc-7d04-429b-be4f-d6d48151b954 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:15.812797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:15.813458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 274/553] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:15.815068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893ca5-3ce9-4b4b-af84-d8946a20f7e4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:15.815354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893ca5-3ce9-4b4b-af84-d8946a20f7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:15.815625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893ca5-3ce9-4b4b-af84-d8946a20f7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:15.836696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6893ca5-3ce9-4b4b-af84-d8946a20f7e4 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:15.838003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6893ca5-3ce9-4b4b-af84-d8946a20f7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:15.853256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e77a68-1983-4920-9cc1-496141b26a27 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:15.896477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:15.897088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:15.897361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:15.897628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:15.898359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:15.898655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:15.900118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:15.915733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 275/554] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:15.916621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6893ca5-3ce9-4b4b-af84-d8946a20f7e4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:15.925355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 280/555] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 31 04:22:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:15.930957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:15.987017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.022386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6893ca5-3ce9-4b4b-af84-d8946a20f7e4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.053959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040b2ce-141a-464f-bcbc-2a8f15227fb7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.059724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a076f4c-20e0-4586-8228-29fa277b256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.060374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076f4c-20e0-4586-8228-29fa277b256f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.060653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076f4c-20e0-4586-8228-29fa277b256f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.060922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076f4c-20e0-4586-8228-29fa277b256f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.072091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a076f4c-20e0-4586-8228-29fa277b256f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:16.074216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a076f4c-20e0-4586-8228-29fa277b256f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:16.090286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6a076f4c-20e0-4586-8228-29fa277b256f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:16.090447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:16.091351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:16.092237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:16.092237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:16.092237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:16.092237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 276/556] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:16.094758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d2e30-a055-4538-9a60-923eb113c995 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.094758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d2e30-a055-4538-9a60-923eb113c995 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.094758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d2e30-a055-4538-9a60-923eb113c995 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.095249 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 277/557] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:16.099957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d2e30-a055-4538-9a60-923eb113c995 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.110629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76413bd0-b5a4-41d4-91fe-05a363c53e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.111278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76413bd0-b5a4-41d4-91fe-05a363c53e82 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.111542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76413bd0-b5a4-41d4-91fe-05a363c53e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.111807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76413bd0-b5a4-41d4-91fe-05a363c53e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.185896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76413bd0-b5a4-41d4-91fe-05a363c53e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:16.187491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76413bd0-b5a4-41d4-91fe-05a363c53e82 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:16.211986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d2e30-a055-4538-9a60-923eb113c995 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.232501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.246914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400d417-1cf5-47da-a499-0d6572cd1e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.247456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400d417-1cf5-47da-a499-0d6572cd1e4b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.247733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400d417-1cf5-47da-a499-0d6572cd1e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.247995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400d417-1cf5-47da-a499-0d6572cd1e4b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.248661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400d417-1cf5-47da-a499-0d6572cd1e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:16.248956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400d417-1cf5-47da-a499-0d6572cd1e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:16.250329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400d417-1cf5-47da-a499-0d6572cd1e4b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:16.262325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 278/558] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:16.264296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895b605-445b-4423-8a87-982c56b7adf6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.264589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895b605-445b-4423-8a87-982c56b7adf6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.264860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895b605-445b-4423-8a87-982c56b7adf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.282299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6895b605-445b-4423-8a87-982c56b7adf6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:16.283526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6895b605-445b-4423-8a87-982c56b7adf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:16.292613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 281/559] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 04:22:16.293872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacae7-79c9-494e-81b2-b6c0105249dd None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.294151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacae7-79c9-494e-81b2-b6c0105249dd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.294421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacae7-79c9-494e-81b2-b6c0105249dd None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.295674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 282/560] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:16.296358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eacae7-79c9-494e-81b2-b6c0105249dd None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.321868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.358189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 279/561] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:16.358904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895b605-445b-4423-8a87-982c56b7adf6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.404169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eacae7-79c9-494e-81b2-b6c0105249dd None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.438946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9993-e31b-4ba3-aaff-af41845b81c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.439577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9993-e31b-4ba3-aaff-af41845b81c3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.439853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9993-e31b-4ba3-aaff-af41845b81c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.440123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9993-e31b-4ba3-aaff-af41845b81c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.451392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c9993-e31b-4ba3-aaff-af41845b81c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8f8c9993-e31b-4ba3-aaff-af41845b81c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:16.457199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:16.458013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 283/562] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:16.459249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9993-e31b-4ba3-aaff-af41845b81c3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.475481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895b605-445b-4423-8a87-982c56b7adf6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.486959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.512559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f43daf-1f5d-4609-a7ba-f008d792d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.513144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43daf-1f5d-4609-a7ba-f008d792d4de None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.513422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43daf-1f5d-4609-a7ba-f008d792d4de None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.513689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43daf-1f5d-4609-a7ba-f008d792d4de None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.520806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f43daf-1f5d-4609-a7ba-f008d792d4de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:16.522580 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f43daf-1f5d-4609-a7ba-f008d792d4de None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-69f43daf-1f5d-4609-a7ba-f008d792d4de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:16.534162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:16.535126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:16.536056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:16.540018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 280/563] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:16.541262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c1cf2-6fa3-4b9d-b0d7-5aa6b7b7a77e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.541630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c1cf2-6fa3-4b9d-b0d7-5aa6b7b7a77e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.541880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c1cf2-6fa3-4b9d-b0d7-5aa6b7b7a77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.546621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.560587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dd4c1cf2-6fa3-4b9d-b0d7-5aa6b7b7a77e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:16.561764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd4c1cf2-6fa3-4b9d-b0d7-5aa6b7b7a77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:16.564054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9993-e31b-4ba3-aaff-af41845b81c3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.599224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.599858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.600131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.600396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.610908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:16.612380 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:16.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:16.629355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:16.630223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:16.630223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:16.630223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:16.630223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 284/564] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:16.630475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 281/565] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:16.631312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c1cf2-6fa3-4b9d-b0d7-5aa6b7b7a77e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.634159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.727145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.732567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c1cf2-6fa3-4b9d-b0d7-5aa6b7b7a77e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.745370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140ba68-5be6-4f29-891c-6161fcd4a9f5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.767682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.768218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.768493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.768757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.769612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:16.769774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:16.771254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:16.781184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.781901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.782270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.782631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.783409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:16.783796 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:16.785268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:16.787826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.792844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 282/566] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:16.792844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.798539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 285/567] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:16.799478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.901530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bb852-174c-45d0-b9d2-bbb1832a892f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.908690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9834b9-0213-4be4-a7f2-0f5726c2c2f4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:16.936811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa58a0-ed3a-4c44-ac57-36478bd5f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.937434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa58a0-ed3a-4c44-ac57-36478bd5f452 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.937706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa58a0-ed3a-4c44-ac57-36478bd5f452 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.937971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa58a0-ed3a-4c44-ac57-36478bd5f452 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.945299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:16.945931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:16.946211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:16.946505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:16.947165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:16.947466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:16.948236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa58a0-ed3a-4c44-ac57-36478bd5f452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:16.949433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-71fa58a0-ed3a-4c44-ac57-36478bd5f452 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:16.952999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:16.954061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:16.955102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:16.959052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 283/568] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:16.960052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa58a0-ed3a-4c44-ac57-36478bd5f452 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.963325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 286/569] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:16.963325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:16.980946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.020718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.062297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73a303-8ab7-46f7-bf45-46522be5275c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.062824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa58a0-ed3a-4c44-ac57-36478bd5f452 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.097179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.097812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.098085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.098356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.098912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.099550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.099829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.100257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.108802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.110272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.111369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.112817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:17.125396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:17.126504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:17.127398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:17.127398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:17.127398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:17.127398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 287/570] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:17.127398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:17.127680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:17.128481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:17.130523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:17.130523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:17.130523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:17.130523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 284/571] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:17.130523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.207047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.239953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437d702-eea9-4c87-b683-067e2257432b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.244223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e688d-146d-461d-a19c-550c929b6a12 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.254960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.273648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.274199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.274468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.274774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.275454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:17.275755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.278363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.279945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3b01d-ea55-4775-b4d0-ac1ebab0fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.280754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3b01d-ea55-4775-b4d0-ac1ebab0fe2d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.281022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3b01d-ea55-4775-b4d0-ac1ebab0fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.281284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3b01d-ea55-4775-b4d0-ac1ebab0fe2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.286745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 285/572] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:17.287550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.293620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3b01d-ea55-4775-b4d0-ac1ebab0fe2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ffe3b01d-ea55-4775-b4d0-ac1ebab0fe2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:17.298947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:17.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 288/573] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:17.300929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3b01d-ea55-4775-b4d0-ac1ebab0fe2d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.409930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d38f99-e624-452d-a942-9424c0d4d411 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.422665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3b01d-ea55-4775-b4d0-ac1ebab0fe2d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.449038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.455459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.456089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.456412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.456674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.459614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.460278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.460551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.460819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.461463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:17.461761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.463210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.466016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.467236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.476009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 289/574] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:17.476843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:17.481661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:17.482524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:17.483430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 286/575] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:17.483912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.508486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.586000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c3b5-6534-4cea-bb79-d88df9a45d7a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.599145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39003d59-3e7c-43fc-a5b6-d5e943f65908 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.623114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.623818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.624128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.624360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.625004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:17.625299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.626684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.639323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 290/576] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:17.640179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.643836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.644479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.644758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.645027 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.656810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.658306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:17.673583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:17.674754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:17.675817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:17.675817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:17.675817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:17.679359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 287/577] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:17.680233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.732490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.759630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.771004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791201c8-3a98-4ed8-a8eb-994d6e769ac1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.799302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76367c1-6613-409c-bc2c-65019ea840cd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.806598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.807294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.807706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.807868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.809071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:17.809370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.811088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.819601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 291/578] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:17.820429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.833726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.834359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.834634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.834939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.843634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.845011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:17.857353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:17.858500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 04:22:17.859766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:17.863376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 288/579] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:17.864145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:17.937573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1ed1f-e4a5-44e7-9509-680e18bd8d35 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.969674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7407cfa-b9fb-40c8-8272-cd36fbfd3f2b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:17.972245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:17.982842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:17.983671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:17.984059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:17.984458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:17.985397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:17.985834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:17.987888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.003156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 292/580] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:18.004032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.016906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.017717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.018097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.018446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.021472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:18.025835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.027398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:18.039404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:18.040476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 289/581] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:18.041367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.123699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8f8f7-f880-4c11-810c-113c1c3eaa0f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.155927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3eaee-37c0-4443-9f4b-b317ef75b825 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.163433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.164059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.164337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.164608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.165644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:18.165944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.167753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.180684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 293/582] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:18.181530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.199336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.199978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.200258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.200540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.211799 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.213268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:18.228169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:18.229330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:18.230219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:18.230219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:18.230219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:18.230219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 290/583] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:18.230635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.250536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:18.267766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:18.298626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520e37a-87e3-4cf1-8d90-9705a78ceacd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.332883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.333496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.333771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.334058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.334841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:18.335144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.335744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8ce82-5562-4da3-a0a9-d534ccaf25d4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.336576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.359818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 294/584] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:18.359818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.374815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.375371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.375643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.375907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.376636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=5e4604fb28234efc9f107cb1f4aa57ad) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:18.376935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.378376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.383739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 291/585] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/role_assignments?user.id=5e4604fb28234efc9f107cb1f4aa57ad&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:18.384504 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.466148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9af614-6059-4a4b-b5b3-9fb7a912601b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.487174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:18.501834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:18.509784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.510505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.510796 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.511074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.511639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356235e7-a208-48fc-bd6e-0376714a4e0c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.511776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:18.512061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.513648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.527983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 295/586] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:18.528423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.560923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.561707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.561982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.562278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.570643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.572255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:18.584778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:18.585748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 292/587] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:18.587350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.639825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d22a6-54f2-4af7-851a-354d2d36a829 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.673649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.674614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.674918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.675189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.676150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:18.676602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.677975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.696673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 296/588] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:18.697300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.699109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd0c4a-1014-457d-8f38-e6846aa469fb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.735767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.736506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.736871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.737217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.747026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:18.749532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.751050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.753208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:18.767227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:18.768182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:18.769032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:18.769032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:18.769032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:18.769032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 293/589] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:18.769247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.812480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ec210-ab72-4c99-932e-0abe1d7a0f73 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.848988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a9cf-01cc-4b61-bb20-e5a67f818bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.849654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a9cf-01cc-4b61-bb20-e5a67f818bd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.849934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a9cf-01cc-4b61-bb20-e5a67f818bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.850211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a9cf-01cc-4b61-bb20-e5a67f818bd2 None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/708ce6cfc5f64d26a418b4339fd883fb/roles/efe57f5de58341b6898d0d4f45a9ffc0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.865034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a9cf-01cc-4b61-bb20-e5a67f818bd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=708ce6cfc5f64d26a418b4339fd883fb, role_id=efe57f5de58341b6898d0d4f45a9ffc0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.866502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a9cf-01cc-4b61-bb20-e5a67f818bd2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.881971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229372bf-a32e-44ed-b66f-c4c8b51d8dfa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:18.886812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 297/590] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:18 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/708ce6cfc5f64d26a418b4339fd883fb/roles/efe57f5de58341b6898d0d4f45a9ffc0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:22:18.888013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a9cf-01cc-4b61-bb20-e5a67f818bd2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:18.929154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:18.930214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:18.930811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:18.931294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:18.932761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:18.933397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:18.935220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:18.961985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 294/591] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:18.963034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.000879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.004579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a9cf-01cc-4b61-bb20-e5a67f818bd2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.040584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.063001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.064045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.064516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.064991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.077922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.079699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:19.093095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:19.094034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 04:22:19.094963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:19.096011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 298/592] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:19.096840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.110186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5594d42-1332-4331-920a-5032f21111b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.153988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.155024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.155314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.155589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.156496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:19.156846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.158879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.174341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 295/593] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:19.179335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.206324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bfb3a-1f77-491e-8cb1-d41ec9a25255 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.247094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.247564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.248043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.259488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.261376 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:19.276662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:19.277611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:19.278480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:19.278480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:19.278480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:19.280613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 299/594] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:19.281669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.297908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.316807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462df47-df91-49e2-8ad2-05cc7fbfc8e2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.331576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.371995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.372624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.372904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.373324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.374218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:19.374524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.376312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.396277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6f3e9-4474-40ac-8436-24eea4b9e0d4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.401156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 296/595] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:19.402036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.434190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.435275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.435745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.436324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.444644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.446548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:19.459724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:19.460694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:19.461554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:19.461554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:19.461554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:19.461554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:19.461554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:19.461554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:19.461554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:19.461554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:19.464484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 300/596] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:19.465715 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.531719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2397d-4b16-488e-8f68-e336fd2f01ff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.559593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.578977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.579830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.580250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.580736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.581519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:19.581986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.583523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.593512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501b75b-ee62-4408-b00b-6f0dd6314a79 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.596936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 297/597] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:19.597928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.628643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.629101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.629854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.630233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.630627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.642352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.644209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:19.658793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:19.659845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:19.660718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:19.660718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:19.660718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:19.660886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 301/598] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:19.662050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.711318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92762c5d-8059-461c-8c08-49325c03b995 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.747976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.748983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.749570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.750011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.750905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:19.751378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.753105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.764586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4901b-608c-4a5d-8c69-a9481e2568c2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.767021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 298/599] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:19.768039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.788576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.800553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.801131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.801406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.801673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.809045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.810522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:19.822659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:19.823777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:19.824824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 302/600] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:19.825366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.871691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.885475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df056-92e3-4df1-aeea-44b58fa6ef51 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.922999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.924088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.924587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.925046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.925979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:19.926471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.928108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.929763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355e89c-79ba-46e5-a831-15ee6a8b0ae2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.941630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 299/601] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:19.943172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.964584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34a836-71b7-4301-a466-90ff7749f4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.965255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34a836-71b7-4301-a466-90ff7749f4ff None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.965527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34a836-71b7-4301-a466-90ff7749f4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.965792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34a836-71b7-4301-a466-90ff7749f4ff None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/ca15f81281dc42f4befeb9335698d18f/roles/efe57f5de58341b6898d0d4f45a9ffc0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.981900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34a836-71b7-4301-a466-90ff7749f4ff None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=ca15f81281dc42f4befeb9335698d18f, role_id=efe57f5de58341b6898d0d4f45a9ffc0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.983457 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34a836-71b7-4301-a466-90ff7749f4ff None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.999537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 303/602] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:19 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/ca15f81281dc42f4befeb9335698d18f/roles/efe57f5de58341b6898d0d4f45a9ffc0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:22:20.000872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34a836-71b7-4301-a466-90ff7749f4ff None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:20.041565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:20.057520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1201dc4-9fa2-490f-9b0c-c2e655b723e9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:20.099357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:20.100502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:20.101037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:20.101503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:20.102456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:20.103022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:20.105079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:20.120299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 300/603] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:20.122810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34a836-71b7-4301-a466-90ff7749f4ff None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:20.126440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:20.138577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:20.160590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:20.161336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:20.161618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:20.161888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:20.169444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:20.171099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:20.184173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:20.185122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:20.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 304/604] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:20.191645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:20.235561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d742b-0677-4c6e-87e1-13d685f4e7bd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:20.283668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7529f-2fe2-470c-9f68-4905937e1eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:20.284467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7529f-2fe2-470c-9f68-4905937e1eca None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:20.284743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7529f-2fe2-470c-9f68-4905937e1eca None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:20.285012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7529f-2fe2-470c-9f68-4905937e1eca None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/6f3a86b76cff4b48a382888cbac913f4/roles/efe57f5de58341b6898d0d4f45a9ffc0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:20.295112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:20.302491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7529f-2fe2-470c-9f68-4905937e1eca None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=6f3a86b76cff4b48a382888cbac913f4, role_id=efe57f5de58341b6898d0d4f45a9ffc0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:20.303288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927d214-7f20-4927-bee8-66a1ee23e13d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:20.303997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7529f-2fe2-470c-9f68-4905937e1eca None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:20.322817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 301/605] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:20 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/6f3a86b76cff4b48a382888cbac913f4/roles/efe57f5de58341b6898d0d4f45a9ffc0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:22:20.349247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b653ca1-843a-4acc-8a65-1890615cfa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:20.350234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b653ca1-843a-4acc-8a65-1890615cfa07 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:20.350759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b653ca1-843a-4acc-8a65-1890615cfa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:20.351229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b653ca1-843a-4acc-8a65-1890615cfa07 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:20.352259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b653ca1-843a-4acc-8a65-1890615cfa07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:20.352743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b653ca1-843a-4acc-8a65-1890615cfa07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:20.355090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b653ca1-843a-4acc-8a65-1890615cfa07 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:20.369895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 305/606] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:20.376673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7529f-2fe2-470c-9f68-4905937e1eca None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:20.390700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fc113-1e46-4cda-82da-245b21a8adb0 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:20.391295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fc113-1e46-4cda-82da-245b21a8adb0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:20.391739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fc113-1e46-4cda-82da-245b21a8adb0 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:20.393572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 306/607] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:20.399497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef5bd7-dbf8-418a-a1c2-8dbd48d26122 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:20.400541 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef5bd7-dbf8-418a-a1c2-8dbd48d26122 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:20.400985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef5bd7-dbf8-418a-a1c2-8dbd48d26122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:20.421224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b0ef5bd7-dbf8-418a-a1c2-8dbd48d26122 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:20.422915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b0ef5bd7-dbf8-418a-a1c2-8dbd48d26122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:20.451849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:20.497395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7529f-2fe2-470c-9f68-4905937e1eca None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:20.502850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 307/608] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:22:20.554992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3227108-54e4-482b-8165-7a9850ba822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:20.555972 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3227108-54e4-482b-8165-7a9850ba822f None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:20.556442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3227108-54e4-482b-8165-7a9850ba822f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:20.556891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3227108-54e4-482b-8165-7a9850ba822f None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/13603c6cd9544f5391256ea600ad0709/roles/efe57f5de58341b6898d0d4f45a9ffc0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:20.576220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3227108-54e4-482b-8165-7a9850ba822f None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=13603c6cd9544f5391256ea600ad0709, role_id=efe57f5de58341b6898d0d4f45a9ffc0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:20.578003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3227108-54e4-482b-8165-7a9850ba822f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:20.598489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 302/609] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:20 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/13603c6cd9544f5391256ea600ad0709/roles/efe57f5de58341b6898d0d4f45a9ffc0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:22:20.800272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef5bd7-dbf8-418a-a1c2-8dbd48d26122 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:20.892616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:20.911516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef5bd7-dbf8-418a-a1c2-8dbd48d26122 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:20.948519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19fadc-7066-4504-8383-a3be208c492d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:20.949455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19fadc-7066-4504-8383-a3be208c492d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:20.949738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19fadc-7066-4504-8383-a3be208c492d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:20.950053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19fadc-7066-4504-8383-a3be208c492d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:20.963574 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19fadc-7066-4504-8383-a3be208c492d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-be19fadc-7066-4504-8383-a3be208c492d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:20.968617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:20.969511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:20.970312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 308/610] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:20.976258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3227108-54e4-482b-8165-7a9850ba822f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:21.070163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23fd0-e364-4a77-8688-930f211619f9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:21.070487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23fd0-e364-4a77-8688-930f211619f9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:21.070743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23fd0-e364-4a77-8688-930f211619f9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:21.072833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 309/611] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:21.079630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6288166-a977-4053-b09c-6a57088a3047 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:21.080162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6288166-a977-4053-b09c-6a57088a3047 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:21.080641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6288166-a977-4053-b09c-6a57088a3047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:21.095132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3227108-54e4-482b-8165-7a9850ba822f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:21.105372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:21.127782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a6288166-a977-4053-b09c-6a57088a3047 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:21.129081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a6288166-a977-4053-b09c-6a57088a3047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:21.138597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:21.143197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:21.143481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:21.143754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:21.144451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:21.144752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:21.148457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:21.165086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 303/612] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:21.171753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:21.259025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 310/613] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:22:21.324898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378835f1-41d8-4502-9da7-227749fcebd5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:21.368612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:21.369252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:21.369528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:21.369851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:21.380997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:21.382462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:21.397094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:21.398043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:21.398936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:21.398936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:21.398936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:21.398936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 304/614] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:21.404425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6288166-a977-4053-b09c-6a57088a3047 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:21.506863 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:21.534164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6288166-a977-4053-b09c-6a57088a3047 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:21.574099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:21.575170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:21.575648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:21.576147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:21.576987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:21.577480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:21.581212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:21.595710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 311/615] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:21.601374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:21.622130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:21.648500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:21.780726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d056cd-6a7e-4b97-820d-e6a0615e2f32 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:21.796434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26083483-ef62-46fe-a9a7-daf9e6d586ab None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:21.844703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:21.845236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:21.845507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:21.845771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:21.855236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:21.856700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:21.860310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd312fd-90fb-433b-b171-b7ccb36c1bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:21.860946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd312fd-90fb-433b-b171-b7ccb36c1bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:21.861223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd312fd-90fb-433b-b171-b7ccb36c1bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:21.861490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd312fd-90fb-433b-b171-b7ccb36c1bf8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:21.865370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:21.877173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:21.878149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:21.879098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:21.880871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 305/616] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:21.885805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd312fd-90fb-433b-b171-b7ccb36c1bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0bd312fd-90fb-433b-b171-b7ccb36c1bf8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:21.891082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:21.893149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:21.895433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:21.896967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:21.897124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 312/617] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:21.901839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd312fd-90fb-433b-b171-b7ccb36c1bf8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:22.026524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd312fd-90fb-433b-b171-b7ccb36c1bf8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:22.075486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:22.091660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37777e-bf92-4cf3-a9d1-b30799a3eeee None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:22.130123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:22.131026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.131555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.132060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.132939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:22.133381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:22.134971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:22.139998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:22.140628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.140906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.141175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.141853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:22.142152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:22.143599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:22.155468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 306/618] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 31 04:22:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:22.160683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:22.165347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 313/619] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:22.170503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:22.269310 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3a5f7-da7e-4a31-92f1-ae8e4deea8fc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:22.301065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:22.308549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:22.309477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.310064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.310671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.322603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:22.324662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:22.329130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:22.348539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:22.349786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:22.350878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:22.350878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:22.350878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:22.361533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 307/620] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:22.364093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:22.406897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22aef4-aa40-4874-801f-25a9f978772d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:22.452560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713e42a-27a7-485b-9272-d9423bb24c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:22.453208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713e42a-27a7-485b-9272-d9423bb24c25 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.453489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713e42a-27a7-485b-9272-d9423bb24c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.453760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713e42a-27a7-485b-9272-d9423bb24c25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.469864 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713e42a-27a7-485b-9272-d9423bb24c25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:22.472133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713e42a-27a7-485b-9272-d9423bb24c25 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:22.477164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2663e-9a62-4b13-846f-3bf195874c5d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b713e42a-27a7-485b-9272-d9423bb24c25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:22.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:22.491055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:22.492324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:22.492324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:22.492324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:22.492324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 314/621] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:22.492803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ba0fa-b99c-4b4f-8d77-f2ada428a0b7 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.493087 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ba0fa-b99c-4b4f-8d77-f2ada428a0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.493363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ba0fa-b99c-4b4f-8d77-f2ada428a0b7 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.494990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 315/622] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:22.500272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ba0fa-b99c-4b4f-8d77-f2ada428a0b7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:22.520472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d1263-f9ce-448a-b55d-c5843955fcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:22.521226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1263-f9ce-448a-b55d-c5843955fcfe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.521576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1263-f9ce-448a-b55d-c5843955fcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.521913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1263-f9ce-448a-b55d-c5843955fcfe None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.522684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1263-f9ce-448a-b55d-c5843955fcfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:22.523082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1263-f9ce-448a-b55d-c5843955fcfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:22.524534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1263-f9ce-448a-b55d-c5843955fcfe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:22.537970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 308/623] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:22.543790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517634-b00c-4270-82d3-f4a46f52969c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.544149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517634-b00c-4270-82d3-f4a46f52969c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.544494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517634-b00c-4270-82d3-f4a46f52969c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.563974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e1517634-b00c-4270-82d3-f4a46f52969c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:22.565253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e1517634-b00c-4270-82d3-f4a46f52969c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:22.614786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:22.632221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ba0fa-b99c-4b4f-8d77-f2ada428a0b7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:22.643005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 309/624] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:22.643005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1517634-b00c-4270-82d3-f4a46f52969c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:22.655075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:22.713499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:22.714226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.714509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.714901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.715572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:22.715874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:22.717277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:22.732432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 316/625] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:22.737634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:22.812984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1517634-b00c-4270-82d3-f4a46f52969c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:22.849524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6e80d-b096-45a5-89ef-c974e6a40694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:22.850264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6e80d-b096-45a5-89ef-c974e6a40694 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.850547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6e80d-b096-45a5-89ef-c974e6a40694 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.850845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6e80d-b096-45a5-89ef-c974e6a40694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.860808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6e80d-b096-45a5-89ef-c974e6a40694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:22.862550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6e80d-b096-45a5-89ef-c974e6a40694 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:22.863189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37aaad5-a09e-464c-b9fc-3480e2cd7fb5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8cc6e80d-b096-45a5-89ef-c974e6a40694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:22.877684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:22.878608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:22.879527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:22.879527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:22.879527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:22.879527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:22.879527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:22.879527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:22.879527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:22.879527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:22.883037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 310/626] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:22.884056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bbe5e-da5a-4d3d-ac37-0b00696bd914 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.884334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bbe5e-da5a-4d3d-ac37-0b00696bd914 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.884605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bbe5e-da5a-4d3d-ac37-0b00696bd914 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.885937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 311/627] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:22.886617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bbe5e-da5a-4d3d-ac37-0b00696bd914 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:22.910245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e35c66-9166-4b20-bcdd-e9c54194fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:22.911120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e35c66-9166-4b20-bcdd-e9c54194fc30 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.911670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e35c66-9166-4b20-bcdd-e9c54194fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.912013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e35c66-9166-4b20-bcdd-e9c54194fc30 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.922153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e35c66-9166-4b20-bcdd-e9c54194fc30 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:22.923860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e35c66-9166-4b20-bcdd-e9c54194fc30 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:22.937475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-49e35c66-9166-4b20-bcdd-e9c54194fc30 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:22.938957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:22.940434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:22.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 317/628] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:22:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:22.942707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925cd4c-c2e3-4e5c-b1e1-f847d72aa3d1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:22.942838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925cd4c-c2e3-4e5c-b1e1-f847d72aa3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:22.943076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925cd4c-c2e3-4e5c-b1e1-f847d72aa3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:22.963577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d925cd4c-c2e3-4e5c-b1e1-f847d72aa3d1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:22.965121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d925cd4c-c2e3-4e5c-b1e1-f847d72aa3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:22.990324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:23.007868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bbe5e-da5a-4d3d-ac37-0b00696bd914 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.039473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 318/629] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:23.040676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925cd4c-c2e3-4e5c-b1e1-f847d72aa3d1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.061990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.061990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.061990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.061990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.062409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:23.062409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:23.063762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:23.086932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 312/630] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:23.088150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.181673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925cd4c-c2e3-4e5c-b1e1-f847d72aa3d1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.207643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d4284-daea-4354-9946-19487053c879 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.209161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:23.218918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.219494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.219779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.220053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.220798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:23.221102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:23.222554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:23.245271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 319/631] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:22:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:23.245271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.254539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc16e2-d6f8-4283-a7c9-1a2e3cf7fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.255201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc16e2-d6f8-4283-a7c9-1a2e3cf7fe93 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.255510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc16e2-d6f8-4283-a7c9-1a2e3cf7fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.255775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc16e2-d6f8-4283-a7c9-1a2e3cf7fe93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.262042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:23.267211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc16e2-d6f8-4283-a7c9-1a2e3cf7fe93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-72dc16e2-d6f8-4283-a7c9-1a2e3cf7fe93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:23.271970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:23.272885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:23.273705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 313/632] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:23.275040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62800e5e-c337-4d3d-b774-1734a0e6a24c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.275040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62800e5e-c337-4d3d-b774-1734a0e6a24c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.275040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62800e5e-c337-4d3d-b774-1734a0e6a24c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.277232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 314/633] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:23.281296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62800e5e-c337-4d3d-b774-1734a0e6a24c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.352172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5e472-d08a-4219-b6e1-e1e9b4d2451d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.387636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc170b-14a1-42a5-8431-c4da19ea9b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.388296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc170b-14a1-42a5-8431-c4da19ea9b0c None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.388570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc170b-14a1-42a5-8431-c4da19ea9b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.388756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62800e5e-c337-4d3d-b774-1734a0e6a24c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.389195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc170b-14a1-42a5-8431-c4da19ea9b0c None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/5e4604fb28234efc9f107cb1f4aa57ad/roles/efe57f5de58341b6898d0d4f45a9ffc0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.403177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc170b-14a1-42a5-8431-c4da19ea9b0c None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=5e4604fb28234efc9f107cb1f4aa57ad, role_id=efe57f5de58341b6898d0d4f45a9ffc0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:23.404623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc170b-14a1-42a5-8431-c4da19ea9b0c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:23.421449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 320/634] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:23 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/5e4604fb28234efc9f107cb1f4aa57ad/roles/efe57f5de58341b6898d0d4f45a9ffc0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:22:23.422273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc170b-14a1-42a5-8431-c4da19ea9b0c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.434754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c76a4e-8ac5-4e5b-9ec2-c96346273bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.435392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76a4e-8ac5-4e5b-9ec2-c96346273bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.435655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76a4e-8ac5-4e5b-9ec2-c96346273bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.435969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76a4e-8ac5-4e5b-9ec2-c96346273bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.474758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:23.517600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:23.517814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c76a4e-8ac5-4e5b-9ec2-c96346273bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:23.519314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c76a4e-8ac5-4e5b-9ec2-c96346273bf7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:23.565605 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc170b-14a1-42a5-8431-c4da19ea9b0c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.604543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5171cc-0ad4-4371-b6d4-8d9f90d2d75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.605166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5171cc-0ad4-4371-b6d4-8d9f90d2d75d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.605444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5171cc-0ad4-4371-b6d4-8d9f90d2d75d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.605718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5171cc-0ad4-4371-b6d4-8d9f90d2d75d None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.606392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5171cc-0ad4-4371-b6d4-8d9f90d2d75d None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:23.606693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5171cc-0ad4-4371-b6d4-8d9f90d2d75d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:23.608096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5171cc-0ad4-4371-b6d4-8d9f90d2d75d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:23.616382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 321/635] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:23.617566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659bcef-ff1e-4c8c-a714-e4600602d80c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.617850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659bcef-ff1e-4c8c-a714-e4600602d80c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.618123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659bcef-ff1e-4c8c-a714-e4600602d80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.636002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f659bcef-ff1e-4c8c-a714-e4600602d80c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:23.637186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f659bcef-ff1e-4c8c-a714-e4600602d80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:23.642067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 315/636] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:23.643189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c76a4e-8ac5-4e5b-9ec2-c96346273bf7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.709165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:23.722782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 322/637] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:23.722782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659bcef-ff1e-4c8c-a714-e4600602d80c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.756040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c76a4e-8ac5-4e5b-9ec2-c96346273bf7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.791469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff9215-c24c-477f-a73c-8c37b22055fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.792169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff9215-c24c-477f-a73c-8c37b22055fa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.792529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff9215-c24c-477f-a73c-8c37b22055fa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.792938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff9215-c24c-477f-a73c-8c37b22055fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.803432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff9215-c24c-477f-a73c-8c37b22055fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:23.807131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d5ff9215-c24c-477f-a73c-8c37b22055fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:23.809316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:23.810364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:23.811212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:23.816735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 316/638] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:23.817973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff9215-c24c-477f-a73c-8c37b22055fa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.854945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659bcef-ff1e-4c8c-a714-e4600602d80c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.890476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e909e19-042c-431b-8220-70976620f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.891203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e909e19-042c-431b-8220-70976620f1de None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.891485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e909e19-042c-431b-8220-70976620f1de None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.891753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e909e19-042c-431b-8220-70976620f1de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.902080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e909e19-042c-431b-8220-70976620f1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:23.903574 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e909e19-042c-431b-8220-70976620f1de None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9e909e19-042c-431b-8220-70976620f1de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:23.919126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:23.921305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:23.923362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:23.923362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:23.923362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:23.926588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 323/639] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:23.926588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e909e19-042c-431b-8220-70976620f1de None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:23.946986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff9215-c24c-477f-a73c-8c37b22055fa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:23.971710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:23.985098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:23.985854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:23.986226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:23.986589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:23.997715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:23.999215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:24.014110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:24.015066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:24.015966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:24.015966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:24.015966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:24.015966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 317/640] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:24.015966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.049124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e909e19-042c-431b-8220-70976620f1de None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.060814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:24.092426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.093048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.093325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.093599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.094284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:24.094585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.096004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.104198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 324/641] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:24.105318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.134409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641d660-f34e-4bce-870c-0cc333a949c4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.171068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.171678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.171960 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.172224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.172878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:24.173176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.174553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.186657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 318/642] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:24.187348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.217249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc683d5a-674c-4a45-bb70-1260dbe8d76f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.223184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:24.253317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42969cd5-da57-4340-8b60-22ce02de90e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.254061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969cd5-da57-4340-8b60-22ce02de90e3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.254338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969cd5-da57-4340-8b60-22ce02de90e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.254603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969cd5-da57-4340-8b60-22ce02de90e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.266209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42969cd5-da57-4340-8b60-22ce02de90e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-42969cd5-da57-4340-8b60-22ce02de90e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:24.271770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:24.272916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 325/643] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:24.273747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42969cd5-da57-4340-8b60-22ce02de90e3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.300305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478911c-9f70-4f1d-9703-fc431d78aa0b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.307587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:24.337932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.338540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.338885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.339136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.339782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:24.340080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.341452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.353024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 319/644] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:24.354058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.386176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42969cd5-da57-4340-8b60-22ce02de90e3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.422639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.423313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.423566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.423836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.435356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.436892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:24.451828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:24.452772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:24.453810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:24.453810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:24.453810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:24.453810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 326/645] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:24.453810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.459461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:24.468289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9a5d8-018c-4933-baf4-01bd03c11c92 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.505862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.506652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.506795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.507049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.514947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.516440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:24.528774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:24.529709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 04:22:24.530677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:24.534512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 320/646] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:24.535301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.552876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:24.571780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363aaba-92c8-4092-8cb8-f7c0e79f8322 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.605379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.605978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.606247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.606511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.607199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:24.607501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.608880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.621973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 327/647] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:24.621973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.646108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e9191-90a8-40c0-98d3-5775cc767e62 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.687903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.688879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.689154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.689417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.696671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.698827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:24.709791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:24.710761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 321/648] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:24.711711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.728948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:24.737028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82a10e-b99a-4595-8812-30d70d035442 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.772245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.773138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.773613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.773882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.774536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:24.774872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.776270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.789164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 328/649] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:24.789805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.799431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:24.823563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7738-58c1-4d6d-b5c7-24c276072690 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.855947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.856554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.856825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.857095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.857847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:24.858116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.859591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.877922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 322/650] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:24.878778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.900343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0174878-22b9-4a9b-bb72-1d130a556091 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:24.937531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:24.938587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:24.939082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:24.939414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:24.949044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:24.951873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:24.953347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:24.968499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:24.969683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:24.970547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:24.970547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:24.970547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:24.970547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 329/651] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:24.970547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:24.987870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4792c3-02d9-4c66-a7f6-de5c002854ab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.026351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.026924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.027251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.027519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.028233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:25.028530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.029965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.034106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:25.042079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 323/652] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:25.042919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.092778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd6776-7fb1-4fa0-bb61-7ab674b145a4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.127533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.128252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.128596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.128872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.136701 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.138230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:25.149940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:25.150919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:25.151906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 330/653] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:25.152501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.159431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809863b2-01e2-49df-b0ba-cd86bfe6a089 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.189609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:25.195749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436036d7-4efd-44c6-b1bc-2acebe7e267b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.196784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436036d7-4efd-44c6-b1bc-2acebe7e267b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.197418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436036d7-4efd-44c6-b1bc-2acebe7e267b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.197941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436036d7-4efd-44c6-b1bc-2acebe7e267b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.257801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefcb9e-2721-4cb1-8035-8c9ab737072e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.267522 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436036d7-4efd-44c6-b1bc-2acebe7e267b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.268959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436036d7-4efd-44c6-b1bc-2acebe7e267b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.276694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:25.295049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.295669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.295950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.296222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.296875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:25.297177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.298566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.310568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 331/654] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:25.311416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.373886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 324/655] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:25.374891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436036d7-4efd-44c6-b1bc-2acebe7e267b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.405245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:25.411920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfed48-ec1a-44ec-9ba0-51ceb4f23036 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.446450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.447129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.447347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.447650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.448413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=708ce6cfc5f64d26a418b4339fd883fb) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:25.448709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.450166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.455897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 332/656] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/role_assignments?user.id=708ce6cfc5f64d26a418b4339fd883fb&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:25.456682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.493597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436036d7-4efd-44c6-b1bc-2acebe7e267b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.512353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:25.533520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b79d4d-16b0-426b-a7f7-83a62e3edeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.534145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b79d4d-16b0-426b-a7f7-83a62e3edeae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.534335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b79d4d-16b0-426b-a7f7-83a62e3edeae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.534611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b79d4d-16b0-426b-a7f7-83a62e3edeae None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.535404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b79d4d-16b0-426b-a7f7-83a62e3edeae None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:25.535710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b79d4d-16b0-426b-a7f7-83a62e3edeae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.537186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b79d4d-16b0-426b-a7f7-83a62e3edeae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.561991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 325/657] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:25.561991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f18916-001c-4311-91a1-9e6d58b9134b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.561991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f18916-001c-4311-91a1-9e6d58b9134b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.561991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f18916-001c-4311-91a1-9e6d58b9134b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.561991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 326/658] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:25.561991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f18916-001c-4311-91a1-9e6d58b9134b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.587644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894330f-1889-4b42-b088-8009c65111fd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.629009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.629650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.629932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.630210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.638154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.639675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:25.652713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:25.653842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 333/659] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:25.655012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.670488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:25.681803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f18916-001c-4311-91a1-9e6d58b9134b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.722196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d2ef3-f056-466b-88a7-8f08bf3fa817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.722863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2ef3-f056-466b-88a7-8f08bf3fa817 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.723144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2ef3-f056-466b-88a7-8f08bf3fa817 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.723416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2ef3-f056-466b-88a7-8f08bf3fa817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.735699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d2ef3-f056-466b-88a7-8f08bf3fa817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.737153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d2ef3-f056-466b-88a7-8f08bf3fa817 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9c4d2ef3-f056-466b-88a7-8f08bf3fa817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:25.753178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:25.754336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:25.755413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:25.755413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:25.755413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:25.755413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 327/660] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:25.755690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f4834-8785-4d2e-9a9c-96a2d9dd83fa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.755978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f4834-8785-4d2e-9a9c-96a2d9dd83fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.756247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f4834-8785-4d2e-9a9c-96a2d9dd83fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.762710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fc2b6-c419-4054-b595-3f6f79f8bfe4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:25.764422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:25.775627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-740f4834-8785-4d2e-9a9c-96a2d9dd83fa None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:25.777039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-740f4834-8785-4d2e-9a9c-96a2d9dd83fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:25.801801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbe27f-5c10-4f70-b560-a36afe8711e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:25.802349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbe27f-5c10-4f70-b560-a36afe8711e9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:25.802627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbe27f-5c10-4f70-b560-a36afe8711e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:25.802931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbe27f-5c10-4f70-b560-a36afe8711e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:25.860190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 328/661] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:25.862006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f4834-8785-4d2e-9a9c-96a2d9dd83fa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.873720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbe27f-5c10-4f70-b560-a36afe8711e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:25.875396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbe27f-5c10-4f70-b560-a36afe8711e9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:25.962598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:25.988273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 334/662] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:25.989526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbe27f-5c10-4f70-b560-a36afe8711e9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:25.991757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.005531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f4834-8785-4d2e-9a9c-96a2d9dd83fa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.042964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.043834 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.044258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.044699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.045593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:26.046055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.047681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.067075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 329/663] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:26.068533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.098888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbe27f-5c10-4f70-b560-a36afe8711e9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.134583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.135269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.135523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.135853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.136569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:26.136875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.138269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.150817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 335/664] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:26.151810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.180610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a969-53e3-4bdf-bb8a-13f12bd7f652 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.211959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.212627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.220065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.220700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.221000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.221272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.232499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.233967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:26.251386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:26.252302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:26.258320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:26.258320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:26.258320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:26.258320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 330/665] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:26.258320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.258627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296dbe8-72a6-44a7-9d76-9c72db38affd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.289455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf6633-bc4b-4d38-8461-3a133f83d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.290065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf6633-bc4b-4d38-8461-3a133f83d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.290336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf6633-bc4b-4d38-8461-3a133f83d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.290796 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf6633-bc4b-4d38-8461-3a133f83d1c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.369083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c212d1-dfd1-49c7-bca8-11ff4cbc5046 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.374432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf6633-bc4b-4d38-8461-3a133f83d1c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.375943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf6633-bc4b-4d38-8461-3a133f83d1c4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.404125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50031-9181-462b-b5fd-88b0c059e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.404756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50031-9181-462b-b5fd-88b0c059e705 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.405030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50031-9181-462b-b5fd-88b0c059e705 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.405293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50031-9181-462b-b5fd-88b0c059e705 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.415453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50031-9181-462b-b5fd-88b0c059e705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-48c50031-9181-462b-b5fd-88b0c059e705 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:26.421440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:26.422383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:26.423271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:26.427422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 331/666] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:26.428122 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50031-9181-462b-b5fd-88b0c059e705 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.445006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.475619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.491119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 336/667] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:26.492599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf6633-bc4b-4d38-8461-3a133f83d1c4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.541588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50031-9181-462b-b5fd-88b0c059e705 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.578314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.579344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.579857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.580147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.588603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.590437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.597313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf6633-bc4b-4d38-8461-3a133f83d1c4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:26.605256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:26.606256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:26.607355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:26.607355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:26.607355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:26.607355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:26.607355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:26.607355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:26.607355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:26.607355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:26.612153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 332/668] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:26.612956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.635196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.635937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.636355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.636756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.637397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:26.637722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.639128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.651000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 337/669] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:26.652001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.678543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.722862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbf98d-dd0f-4ddb-9b49-c1909705ecc4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.724362 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.758724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.759354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.759637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.759908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.760570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:26.760959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.761906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ce299-791c-4f9e-968c-2c319777f3f6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.762314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.774859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 333/670] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:26.775699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.800271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.801312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.801675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.801870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.812620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.814084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:26.829797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:26.830747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:26.831727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:26.831727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:26.831727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:26.831727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 338/671] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:26.831727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.888233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763ea5-7ff0-4d64-bb8b-3a5aae4cc8bb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.914944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.925422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.926035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.926317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.926590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.927327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:26.927634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.929198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.940920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2da29-000f-432d-bc3d-b1e0d27c08ff None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:26.947478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 334/672] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:26.947478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:26.974875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:26.976217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:26.976907 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:26.977173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:26.977433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:26.984647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:26.986100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:26.997686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:26.998593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:26.999502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:26.999502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:26.999502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:26.999502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:26.999502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:26.999502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:26.999502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:26.999502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:27.003565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 339/673] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:27.004234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.070277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f96dc-e8f7-40bd-96e1-4f7d3e0bba99 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.108786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.109539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.109798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.110075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.112250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce712f-a2b3-4d92-aeae-87f5f4d57abe None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.121075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.122657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:27.139344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:27.140272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:27.141140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:27.141140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:27.141140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:27.141140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 335/674] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:27.141140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.153531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.154167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.154442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.155066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.155707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:27.155999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.157388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.158864 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:27.171244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 340/675] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:27.172148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.228676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:27.252680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f80c9c-8e05-48ee-9040-617b759105d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.273733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df159bf-832d-462b-9631-9ff36bd452a3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.286409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.287134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.287413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.287680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.288410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=ca15f81281dc42f4befeb9335698d18f) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:27.288714 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.290203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.306759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 336/676] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/role_assignments?user.id=ca15f81281dc42f4befeb9335698d18f&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:27.306759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.323053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.323902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.324336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.324566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.325397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:27.325767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.327749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.348060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 341/677] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:27.348060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.412738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:27.438747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b14ac-df01-4bc3-aaa3-182272e9d4e2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.465994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bdc8a-8bf5-4866-9de8-175854672401 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.478243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.479059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.479188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.479606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.480944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:27.480944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.481802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.491195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:27.499103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 337/678] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:27.500390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.513635 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951e66b-e21b-48bc-9dfb-149d73106752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.514270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e66b-e21b-48bc-9dfb-149d73106752 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.514555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e66b-e21b-48bc-9dfb-149d73106752 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.514854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e66b-e21b-48bc-9dfb-149d73106752 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.521913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951e66b-e21b-48bc-9dfb-149d73106752 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.523400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951e66b-e21b-48bc-9dfb-149d73106752 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d951e66b-e21b-48bc-9dfb-149d73106752 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:27.535334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:27.536427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:27.537497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 342/679] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:27.538039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1694c-fd40-4da2-a478-0e1046f676ed None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.538256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1694c-fd40-4da2-a478-0e1046f676ed None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.538530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1694c-fd40-4da2-a478-0e1046f676ed None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.539760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 343/680] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:27.540435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1694c-fd40-4da2-a478-0e1046f676ed None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.622735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d202bc-5d71-4e67-a3a6-ba6b1e4bb789 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.651890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1694c-fd40-4da2-a478-0e1046f676ed None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.660369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.660975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.661254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.661518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.662240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=6f3a86b76cff4b48a382888cbac913f4) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:27.662541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.664126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.667251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:27.670410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 338/681] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/role_assignments?user.id=6f3a86b76cff4b48a382888cbac913f4&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:27.671357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.700733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcc84f-0da8-42c2-af96-9d37095f8692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.701383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcc84f-0da8-42c2-af96-9d37095f8692 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.701658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcc84f-0da8-42c2-af96-9d37095f8692 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.701925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcc84f-0da8-42c2-af96-9d37095f8692 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.710903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcc84f-0da8-42c2-af96-9d37095f8692 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.712395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcc84f-0da8-42c2-af96-9d37095f8692 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c3dcc84f-0da8-42c2-af96-9d37095f8692 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:27.725856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:27.726906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:27.727838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:27.731230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 344/682] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:27.732603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62c126-3c04-486e-b030-31842119e478 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.732882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62c126-3c04-486e-b030-31842119e478 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.733149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62c126-3c04-486e-b030-31842119e478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.756755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:27.757444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fc62c126-3c04-486e-b030-31842119e478 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:27.758771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fc62c126-3c04-486e-b030-31842119e478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:27.800315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c505a-3504-4acd-b0ba-3f86f0a6841e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.839755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.840082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 345/683] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:27.840322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.840595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.840829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62c126-3c04-486e-b030-31842119e478 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.841002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.841553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:27.841850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.843340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:27.857582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 339/684] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:27.868397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:27.953656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62c126-3c04-486e-b030-31842119e478 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.956016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:27.975756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ee55f-4220-4943-93fe-6e74dae578d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:27.991204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:27.991825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:27.992100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:27.992369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:27.993107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:27.993499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:27.995069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:28.001044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:28.014143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:28.014769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:28.015049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:28.015313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:28.015602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 346/685] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:28.016046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=13603c6cd9544f5391256ea600ad0709) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:28.016344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:28.017786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:28.024292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:28.024452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 340/686] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:27 2022] GET /identity/v3/role_assignments?user.id=13603c6cd9544f5391256ea600ad0709&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:28.135881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bed933-d26d-4c5a-9129-fed0741d4701 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:28.158894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:28.180696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0b93-4065-4e5c-a5a5-899f8dbe93ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:28.181344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0b93-4065-4e5c-a5a5-899f8dbe93ef None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:28.181619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0b93-4065-4e5c-a5a5-899f8dbe93ef None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:28.181890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0b93-4065-4e5c-a5a5-899f8dbe93ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:28.253548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:28.300357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa0b93-4065-4e5c-a5a5-899f8dbe93ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:28.302130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa0b93-4065-4e5c-a5a5-899f8dbe93ef None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:28.304676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14a51-cc55-4a00-ac46-ca0345ac9d1a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:28.344523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba7aeb-ca2c-48d9-88e4-09a04ab594f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:28.345164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba7aeb-ca2c-48d9-88e4-09a04ab594f7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:28.345509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba7aeb-ca2c-48d9-88e4-09a04ab594f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:28.345777 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba7aeb-ca2c-48d9-88e4-09a04ab594f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:28.358287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba7aeb-ca2c-48d9-88e4-09a04ab594f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-79ba7aeb-ca2c-48d9-88e4-09a04ab594f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:28.364136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:28.365066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:28.365888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 341/687] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:28.373017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba7aeb-ca2c-48d9-88e4-09a04ab594f7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:28.453567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 347/688] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 04:22:28.461049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0b93-4065-4e5c-a5a5-899f8dbe93ef None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:28.466548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:28.536965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba7aeb-ca2c-48d9-88e4-09a04ab594f7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:28.580901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:28.605371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:28.606299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:28.606743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:28.607247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:28.609682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:28.610146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:28.611688 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:28.628227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 342/689] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:28.634451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:28.655074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0b93-4065-4e5c-a5a5-899f8dbe93ef None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:28.707533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:28.708461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:28.708891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:28.709306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:28.721682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:28.723397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:28.752421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:28.753597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:28.754488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:28.754488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:28.754488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:28.764133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 348/690] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:28.781242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:28.836059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf31a04-b2b8-44a2-829a-de983620cb8e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:28.858623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:28.927331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9118ff-e467-4d11-bcb7-04b25f9db5e3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:28.943084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:28.943425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:28.944147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:28.944355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:28.944631 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:28.969078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:28.969948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:28.972526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:28.973145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:28.973419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:28.973687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:28.974341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:28.974637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:28.976039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:28.998739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:28.999771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:29.000627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:29.000627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:29.000627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:29.000627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 343/692] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:29.004494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:29.006800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 349/692] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:29.012083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:29.198575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff082f72-f9d7-4e2d-8d37-c2e5099a134b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:29.202853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c55439-b850-45e2-827b-dc0c568907a4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:29.262461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:29.264152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:29.264429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:29.264697 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:29.274987 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:29.275609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:29.275882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:29.276214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:29.277373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:29.277669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:29.279090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:29.281338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:29.283155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:29.300787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:29.301712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:29.302588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:29.302588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:29.302588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:29.302588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:29.302588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:29.302588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:29.302588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:29.302588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:29.304579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 350/693] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:29.309847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:29.314755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 344/694] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:29.319837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:29.347912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:29.368442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:29.423462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07071e2d-6fb4-488f-9a3e-c2dc61548d32 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:29.467541 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94d31a-518f-4262-ba35-f4931a8a63d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:29.468940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d31a-518f-4262-ba35-f4931a8a63d2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:29.469412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d31a-518f-4262-ba35-f4931a8a63d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:29.469866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d31a-518f-4262-ba35-f4931a8a63d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:29.471190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94d31a-518f-4262-ba35-f4931a8a63d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:29.471939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94d31a-518f-4262-ba35-f4931a8a63d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:29.473564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94d31a-518f-4262-ba35-f4931a8a63d2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:29.473720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda56e4-1f56-4596-9d84-9930d4b9ebf2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:29.493229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 351/695] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:29.493229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ec71-4a5b-4734-b5f3-f33e31318b7d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:29.493229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ec71-4a5b-4734-b5f3-f33e31318b7d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:29.493229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ec71-4a5b-4734-b5f3-f33e31318b7d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:29.494795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 352/696] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:29.496206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620ec71-4a5b-4734-b5f3-f33e31318b7d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:29.519687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08b627-e5c1-45cb-9f5f-855e4a6736ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:29.520319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b627-e5c1-45cb-9f5f-855e4a6736ef None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:29.520589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b627-e5c1-45cb-9f5f-855e4a6736ef None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:29.520852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b627-e5c1-45cb-9f5f-855e4a6736ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:29.528645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08b627-e5c1-45cb-9f5f-855e4a6736ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:29.530102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08b627-e5c1-45cb-9f5f-855e4a6736ef None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7a08b627-e5c1-45cb-9f5f-855e4a6736ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:29.542836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:29.543758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:29.545289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 345/697] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:22:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:29.545894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447ef1a-8aa4-4167-8256-fa36fa4862e4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:29.545894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447ef1a-8aa4-4167-8256-fa36fa4862e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:29.545894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447ef1a-8aa4-4167-8256-fa36fa4862e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:29.566098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8447ef1a-8aa4-4167-8256-fa36fa4862e4 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:29.567358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8447ef1a-8aa4-4167-8256-fa36fa4862e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:29.601598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:29.610473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620ec71-4a5b-4734-b5f3-f33e31318b7d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:29.646810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 346/698] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:29.648384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447ef1a-8aa4-4167-8256-fa36fa4862e4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:29.656117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:29.657006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:29.657478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:29.657964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:29.658931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=5e4604fb28234efc9f107cb1f4aa57ad) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:29.659448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:29.661171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:29.674879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 353/699] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:29 2022] GET /identity/v3/role_assignments?user.id=5e4604fb28234efc9f107cb1f4aa57ad&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:29.675247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:29.798534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447ef1a-8aa4-4167-8256-fa36fa4862e4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:29.836577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:29.837194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:29.837467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:29.837732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:29.838479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:29.838808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:29.840510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:29.860226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 347/700] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:22:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:29.867480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:29.980963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9d99d-f250-41b3-ab0b-f797c0b236f5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:30.015891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3f7f2-b829-40f9-b94f-90583edbb99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:30.016762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3f7f2-b829-40f9-b94f-90583edbb99f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.017181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3f7f2-b829-40f9-b94f-90583edbb99f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.017621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3f7f2-b829-40f9-b94f-90583edbb99f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.045695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:30.056331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:30.094549 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3f7f2-b829-40f9-b94f-90583edbb99f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:30.096065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3f7f2-b829-40f9-b94f-90583edbb99f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:30.135851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:30.171883 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5111cf-d235-42cc-b041-ade34dd92d08 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:30.205507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 354/701] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:30.206847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d665092-0887-480b-8e0c-6a2902f653bc None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.207137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d665092-0887-480b-8e0c-6a2902f653bc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.207412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d665092-0887-480b-8e0c-6a2902f653bc None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.214784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 355/702] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:30.214784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d665092-0887-480b-8e0c-6a2902f653bc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:30.217929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d69b3-dda6-4e54-9cc5-8581cb24894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:30.219007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d69b3-dda6-4e54-9cc5-8581cb24894e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.219446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d69b3-dda6-4e54-9cc5-8581cb24894e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.219726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d69b3-dda6-4e54-9cc5-8581cb24894e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.234737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d69b3-dda6-4e54-9cc5-8581cb24894e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:30.237118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d69b3-dda6-4e54-9cc5-8581cb24894e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-459d69b3-dda6-4e54-9cc5-8581cb24894e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:30.258906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:30.259886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:30.260740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:30.260740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:30.260740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:30.260740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 348/703] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:30.262415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44631fdb-7a0b-497a-8a05-e12618dc877a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.265353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44631fdb-7a0b-497a-8a05-e12618dc877a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.265353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44631fdb-7a0b-497a-8a05-e12618dc877a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.281678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-44631fdb-7a0b-497a-8a05-e12618dc877a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:30.282890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-44631fdb-7a0b-497a-8a05-e12618dc877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:30.287547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:30.338363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d665092-0887-480b-8e0c-6a2902f653bc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:30.370746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 349/704] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:30.370746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44631fdb-7a0b-497a-8a05-e12618dc877a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:30.388229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3f15-332d-455e-bfe3-4cc3151dd836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:30.388879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3f15-332d-455e-bfe3-4cc3151dd836 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.389154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3f15-332d-455e-bfe3-4cc3151dd836 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.389427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3f15-332d-455e-bfe3-4cc3151dd836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.404795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c3f15-332d-455e-bfe3-4cc3151dd836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:30.406742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c3f15-332d-455e-bfe3-4cc3151dd836 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-db1c3f15-332d-455e-bfe3-4cc3151dd836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:30.425603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:30.426610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:30.428123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:30.428123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:30.428123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:30.430918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 356/705] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:30.433780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720191a3-6961-43f0-b3e7-0c03a40993f0 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.433926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720191a3-6961-43f0-b3e7-0c03a40993f0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.434220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720191a3-6961-43f0-b3e7-0c03a40993f0 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.435686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 357/706] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:30.438540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720191a3-6961-43f0-b3e7-0c03a40993f0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:30.445764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:30.502537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44631fdb-7a0b-497a-8a05-e12618dc877a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:30.543054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a3a90-4324-4791-97bc-10e8915372fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:30.544267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a3a90-4324-4791-97bc-10e8915372fb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.544267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a3a90-4324-4791-97bc-10e8915372fb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.544588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a3a90-4324-4791-97bc-10e8915372fb None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.545105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a3a90-4324-4791-97bc-10e8915372fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:30.545467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a3a90-4324-4791-97bc-10e8915372fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:30.546989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a3a90-4324-4791-97bc-10e8915372fb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:30.561003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 350/707] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:30.562529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbd3a-544f-4e30-952b-1ed8b7d360f7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.562894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbd3a-544f-4e30-952b-1ed8b7d360f7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.563230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbd3a-544f-4e30-952b-1ed8b7d360f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.583297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1a2bbd3a-544f-4e30-952b-1ed8b7d360f7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:30.584953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1a2bbd3a-544f-4e30-952b-1ed8b7d360f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:30.592337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720191a3-6961-43f0-b3e7-0c03a40993f0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:30.609669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:30.647422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:30.648044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.648322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.648592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.649309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:30.649616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:30.651021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:30.668203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 358/708] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:30.668203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:30.668490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 351/709] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:30.668900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bbd3a-544f-4e30-952b-1ed8b7d360f7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:30.721809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:30.787980 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91caa3-e2a5-4896-83a6-112218e6c8b8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:30.791452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bbd3a-544f-4e30-952b-1ed8b7d360f7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:30.832544 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053bb4d-8d06-456c-a729-b0b2b760c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:30.833160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053bb4d-8d06-456c-a729-b0b2b760c87a None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.833506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053bb4d-8d06-456c-a729-b0b2b760c87a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.833781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053bb4d-8d06-456c-a729-b0b2b760c87a None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.834520 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053bb4d-8d06-456c-a729-b0b2b760c87a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:30.835188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:30.836037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:30.836141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053bb4d-8d06-456c-a729-b0b2b760c87a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:30.836627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:30.837112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:30.837887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:30.839599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:30.851055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:30.852104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:30.852976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:30.857193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 352/710] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Aug 31 04:22:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:30.857916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:30.871273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 359/711] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:22:30 2022] POST /identity/v3/users => generated 254 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:30.873122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053bb4d-8d06-456c-a729-b0b2b760c87a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:30.909358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:30.998174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053bb4d-8d06-456c-a729-b0b2b760c87a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:31.012592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b43d8-86c3-4751-9e92-72b6a845d951 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:31.023591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:31.043764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:31.045057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:31.045578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:31.046241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:31.047582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:31.048311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:31.050538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:31.057575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 360/712] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 31 04:22:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:31.058893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:31.067827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:31.068503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:31.070207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:31.070207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:31.078316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:31.079901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:31.093680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:31.094787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:31.095693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 353/713] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:31.096225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:31.182021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:31.194513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd53b7-c9a9-4841-94a4-1f33c9482230 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:31.232800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4f567-3ae6-4887-8360-a2fcb423bbca None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:31.233275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:31.233921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:31.234197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:31.234469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] PATH_INFO: `/v3/services/network` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:31.235100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:31.237461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:31.251524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:31.252583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:31.253429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 361/714] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:31 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:31.254195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:31.281677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:31.282878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:31.283301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:31.283948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:31.285165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:31.285860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:31.288095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:31.297371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 354/715] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 31 04:22:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:31.299338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:31.373844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:31.453540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0740cf-0bed-4d0c-af54-b11aad8c5576 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:31.498515 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b0271-405d-4e34-91c4-10ea9a919396 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:31.502555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:31.503408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:31.503880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:31.504323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:31.505214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:31.505692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:31.507374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:31.520203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 362/716] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:31.526125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:31.544079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:31.547110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:31.547984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:31.548450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:31.548907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:31.550134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:31.550702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:31.552337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:31.558355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 355/717] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:31.564351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:31.638875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e345b11-e2a3-4420-a833-7a911d90b090 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:31.679486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810161de-8670-447b-8437-d5bdebd32b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:31.680116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810161de-8670-447b-8437-d5bdebd32b83 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:31.680396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810161de-8670-447b-8437-d5bdebd32b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:31.680691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810161de-8670-447b-8437-d5bdebd32b83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:31.681434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810161de-8670-447b-8437-d5bdebd32b83 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=708ce6cfc5f64d26a418b4339fd883fb) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:31.681741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810161de-8670-447b-8437-d5bdebd32b83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:31.683606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810161de-8670-447b-8437-d5bdebd32b83 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:31.704866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 363/718] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:31 2022] GET /identity/v3/role_assignments?user.id=708ce6cfc5f64d26a418b4339fd883fb&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=241e99d365564c0fa95cb712f4004785 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:31.705183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:31.714028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaad75e-13b3-4db6-ae7a-2a6fc448b4c7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:31.781381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bfbf3-10f4-42c5-88f6-11038f80f65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:31.782048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfbf3-10f4-42c5-88f6-11038f80f65d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:31.782330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfbf3-10f4-42c5-88f6-11038f80f65d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:31.782606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfbf3-10f4-42c5-88f6-11038f80f65d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:31.783274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bfbf3-10f4-42c5-88f6-11038f80f65d None admin] RBAC: Adding query filter params (type=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:31.783569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bfbf3-10f4-42c5-88f6-11038f80f65d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:31.784949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bfbf3-10f4-42c5-88f6-11038f80f65d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:31.795089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 356/719] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:32.168803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb62f6-fa37-4c8b-86ae-941bf509ee7d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:32.169154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb62f6-fa37-4c8b-86ae-941bf509ee7d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:32.169370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb62f6-fa37-4c8b-86ae-941bf509ee7d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:32.171235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 364/720] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:32.178230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad57775-b40c-4547-813f-89b14fa3a97c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:32.178519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad57775-b40c-4547-813f-89b14fa3a97c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:32.178820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad57775-b40c-4547-813f-89b14fa3a97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:32.203527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ad57775-b40c-4547-813f-89b14fa3a97c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:32.204714 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ad57775-b40c-4547-813f-89b14fa3a97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:32.272620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 357/721] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:32.632141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb62f6-fa37-4c8b-86ae-941bf509ee7d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:32.732303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:32.764686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb62f6-fa37-4c8b-86ae-941bf509ee7d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:32.811542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfb306-6761-4d5e-acad-f57264561d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:32.812197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb306-6761-4d5e-acad-f57264561d0d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:32.812478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb306-6761-4d5e-acad-f57264561d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:32.812751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb306-6761-4d5e-acad-f57264561d0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:32.829770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfb306-6761-4d5e-acad-f57264561d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-96dfb306-6761-4d5e-acad-f57264561d0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:32.835350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:32.836250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:32.837465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 365/722] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:32.844569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad57775-b40c-4547-813f-89b14fa3a97c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:32.957078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:32.975111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad57775-b40c-4547-813f-89b14fa3a97c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:33.028852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d10a7-607f-4c15-a779-10725bc2a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:33.029474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d10a7-607f-4c15-a779-10725bc2a702 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.029751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d10a7-607f-4c15-a779-10725bc2a702 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.030021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d10a7-607f-4c15-a779-10725bc2a702 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.030693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d10a7-607f-4c15-a779-10725bc2a702 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:33.031012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d10a7-607f-4c15-a779-10725bc2a702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:33.032387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d10a7-607f-4c15-a779-10725bc2a702 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:33.046857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 358/723] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:33.055977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfb306-6761-4d5e-acad-f57264561d0d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:33.203483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50476685-4dd2-44f8-b77d-c9991f7fdb61 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.203827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50476685-4dd2-44f8-b77d-c9991f7fdb61 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.204031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50476685-4dd2-44f8-b77d-c9991f7fdb61 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.205591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 359/724] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:33.210754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c6966-241e-4684-abcb-2c207c111f2c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.210995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c6966-241e-4684-abcb-2c207c111f2c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.211265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c6966-241e-4684-abcb-2c207c111f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.228555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfb306-6761-4d5e-acad-f57264561d0d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:33.250846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-584c6966-241e-4684-abcb-2c207c111f2c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:33.252064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-584c6966-241e-4684-abcb-2c207c111f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:33.266178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:33.266849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.267127 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.267394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.275308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:33.280280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:33.281810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:33.297433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:33.299042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:33.300505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:33.300505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:33.300505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:33.300505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 366/725] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:33.309470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:33.394488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 360/726] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:22:33.469073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea87f62-4adb-4e43-9ed4-8693ebc7f638 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:33.503113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0a365-71dd-4ccc-bef0-9c9414579278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:33.503727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a365-71dd-4ccc-bef0-9c9414579278 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.504001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a365-71dd-4ccc-bef0-9c9414579278 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.504269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a365-71dd-4ccc-bef0-9c9414579278 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.504914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0a365-71dd-4ccc-bef0-9c9414579278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:33.505209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0a365-71dd-4ccc-bef0-9c9414579278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:33.506579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0a365-71dd-4ccc-bef0-9c9414579278 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:33.518791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 367/727] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:33.524289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c6966-241e-4684-abcb-2c207c111f2c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:33.652069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:33.684646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c6966-241e-4684-abcb-2c207c111f2c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:33.736451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:33.737450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.737924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.738408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.749934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d746a-7a02-4166-b77d-47ee6318038a None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.750414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d746a-7a02-4166-b77d-47ee6318038a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.750489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d746a-7a02-4166-b77d-47ee6318038a None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.752144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 368/728] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:33.757263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28035ee-9905-496c-8e94-3c03901d6ba1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.758010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28035ee-9905-496c-8e94-3c03901d6ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.758920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:33.760569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:33.761368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28035ee-9905-496c-8e94-3c03901d6ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:33.781144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:33.782128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:33.783081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:33.795619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 361/729] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:33.800199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c28035ee-9905-496c-8e94-3c03901d6ba1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:33.801238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:33.803026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c28035ee-9905-496c-8e94-3c03901d6ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:33.860859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:33.889918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 369/730] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 04:22:33.890894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28035ee-9905-496c-8e94-3c03901d6ba1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:33.936767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771896cd-f23e-4b8d-a5a8-9d469799a711 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:33.984055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:33.984688 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:33.985006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:33.985284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:33.986035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:33.986336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:33.987862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:34.000996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28035ee-9905-496c-8e94-3c03901d6ba1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.002508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:34.011289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 362/731] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 31 04:22:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:34.017069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.044384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d66f97-1332-4a35-a896-253c9394cdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.045683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d66f97-1332-4a35-a896-253c9394cdff None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.046402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d66f97-1332-4a35-a896-253c9394cdff None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.046923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d66f97-1332-4a35-a896-253c9394cdff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.071282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d66f97-1332-4a35-a896-253c9394cdff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-09d66f97-1332-4a35-a896-253c9394cdff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:34.076301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:34.077178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:34.078049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 370/732] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:34.084850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d66f97-1332-4a35-a896-253c9394cdff None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.129048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:34.154827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798a86d-cd24-43cb-8803-6980bac90414 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.191394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d66f97-1332-4a35-a896-253c9394cdff None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.199708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61b72b-b250-43d2-b636-bcfffa417270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.200326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61b72b-b250-43d2-b636-bcfffa417270 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.200669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61b72b-b250-43d2-b636-bcfffa417270 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.200945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61b72b-b250-43d2-b636-bcfffa417270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.237957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec9e5a-a058-4a4e-a9e8-b7420f5b72fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.239328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec9e5a-a058-4a4e-a9e8-b7420f5b72fa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.239610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec9e5a-a058-4a4e-a9e8-b7420f5b72fa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.239882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec9e5a-a058-4a4e-a9e8-b7420f5b72fa None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.240565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec9e5a-a058-4a4e-a9e8-b7420f5b72fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:34.240892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec9e5a-a058-4a4e-a9e8-b7420f5b72fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:34.242295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec9e5a-a058-4a4e-a9e8-b7420f5b72fa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:34.254868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 371/733] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:34.256203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca5ea8-fea7-4832-829b-7ad34897a9ad None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.256493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca5ea8-fea7-4832-829b-7ad34897a9ad None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.256773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca5ea8-fea7-4832-829b-7ad34897a9ad None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.259727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 372/734] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:34.260563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca5ea8-fea7-4832-829b-7ad34897a9ad None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.280186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:34.284568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61b72b-b250-43d2-b636-bcfffa417270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:34.286172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61b72b-b250-43d2-b636-bcfffa417270 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:34.365869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca5ea8-fea7-4832-829b-7ad34897a9ad None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.367448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:34.387256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 363/735] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 04:22:34.388804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c5307-6148-4be3-b67f-8428df165ca9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.389083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c5307-6148-4be3-b67f-8428df165ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.389350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c5307-6148-4be3-b67f-8428df165ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.401960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.402626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.402937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.403208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.406588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ad2c5307-6148-4be3-b67f-8428df165ca9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:34.407792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad2c5307-6148-4be3-b67f-8428df165ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:34.414109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:34.415575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:34.430771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:34.431781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:34.432717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:34.432717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:34.432717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:34.433135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 373/736] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:34.434038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.476223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 364/737] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:34.477620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e487-52c1-4aea-ba2a-2e991367a79c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.477908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e487-52c1-4aea-ba2a-2e991367a79c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.478178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e487-52c1-4aea-ba2a-2e991367a79c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.479440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 365/738] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:34.480154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e487-52c1-4aea-ba2a-2e991367a79c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.515003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:34.546053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c00b8-d892-4ab6-a3ff-16599bb6c598 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.583383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e1b2a-503d-4228-be79-266345272a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.584323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e1b2a-503d-4228-be79-266345272a48 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.584600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e1b2a-503d-4228-be79-266345272a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.585022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e1b2a-503d-4228-be79-266345272a48 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.585976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e1b2a-503d-4228-be79-266345272a48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:34.587392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e1b2a-503d-4228-be79-266345272a48 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:34.595670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e487-52c1-4aea-ba2a-2e991367a79c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.610793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 374/739] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:22:34 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:34.610793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e1b2a-503d-4228-be79-266345272a48 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.645115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:34.653754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498d13a-a45a-4bcb-b0af-fdbea2dedb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.654414 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498d13a-a45a-4bcb-b0af-fdbea2dedb22 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.654963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498d13a-a45a-4bcb-b0af-fdbea2dedb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.655431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498d13a-a45a-4bcb-b0af-fdbea2dedb22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.670101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498d13a-a45a-4bcb-b0af-fdbea2dedb22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:34.671758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498d13a-a45a-4bcb-b0af-fdbea2dedb22 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5498d13a-a45a-4bcb-b0af-fdbea2dedb22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:34.689206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:34.690235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:34.691747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:34.691747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:34.691747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:34.691747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 366/740] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:34.692116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c897b-6ad6-4e2e-b15c-5d7870fcd3f9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.692298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c897b-6ad6-4e2e-b15c-5d7870fcd3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.692918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c897b-6ad6-4e2e-b15c-5d7870fcd3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.715226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-836c897b-6ad6-4e2e-b15c-5d7870fcd3f9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:34.716447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-836c897b-6ad6-4e2e-b15c-5d7870fcd3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:34.768554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e1b2a-503d-4228-be79-266345272a48 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.798366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 367/741] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 04:22:34.799664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c897b-6ad6-4e2e-b15c-5d7870fcd3f9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.809309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:34.813000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.814065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.814379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.814693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.815590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:34.815895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:34.818352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:34.833653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 375/742] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:34.834899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.910865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c897b-6ad6-4e2e-b15c-5d7870fcd3f9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.923743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:34.950834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb456b-b2a7-443c-a63e-757fa9151526 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:34.958855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.959473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.959745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.960525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:34.961192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:34.961488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:34.963340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:34.980104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 368/743] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:34.981624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:34.992456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8644db9-966d-4e44-9dcf-d950890f521f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:34.993093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8644db9-966d-4e44-9dcf-d950890f521f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:34.993368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8644db9-966d-4e44-9dcf-d950890f521f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:34.993634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8644db9-966d-4e44-9dcf-d950890f521f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.007873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8644db9-966d-4e44-9dcf-d950890f521f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f8644db9-966d-4e44-9dcf-d950890f521f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:35.015402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:35.016309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 376/744] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:35.017739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8644db9-966d-4e44-9dcf-d950890f521f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.067003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:35.113894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c09e70-20d8-4296-8cf2-1cacfeb931f1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.142179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8644db9-966d-4e44-9dcf-d950890f521f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.160756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.161496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.161832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.162164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.171013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.172463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.177738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.178380 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.178653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.178950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.181011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:35.183964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:35.185163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 369/745] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:22:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:35.186752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.190587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.192103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:35.207671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:35.208559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:35.209458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:35.209458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:35.209458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:35.209458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:35.209458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:35.209458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:35.209458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:35.209458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:35.210559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 377/746] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:35.211445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.304546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99852e26-2d1b-458d-a43c-923e442ff437 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.314188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402dd1e9-8ceb-457b-8400-ff6448c50d5f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.336826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:35.347906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.348700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.349128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.349545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.350758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:35.351230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.352240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e52f4-5ee8-4ea7-8b6c-1604ea07aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.352877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e52f4-5ee8-4ea7-8b6c-1604ea07aaf8 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.353030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.353176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e52f4-5ee8-4ea7-8b6c-1604ea07aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.353417 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e52f4-5ee8-4ea7-8b6c-1604ea07aaf8 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.354058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e52f4-5ee8-4ea7-8b6c-1604ea07aaf8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.355464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e52f4-5ee8-4ea7-8b6c-1604ea07aaf8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.366845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 370/747] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:35.366845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.373314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 378/748] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:22:35 2022] POST /identity/v3/services => generated 240 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:35.374198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e52f4-5ee8-4ea7-8b6c-1604ea07aaf8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.439381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:35.502856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971afcb7-1d9f-44b3-af46-3a8efcdf8e7d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.504970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e52f4-5ee8-4ea7-8b6c-1604ea07aaf8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.542658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.543362 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.543643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.543912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.544657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:35.544959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.546435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.551811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.552625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.553051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.553470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.554392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:35.554866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.556745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.568193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 371/749] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:35.569148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.573634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 379/750] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:35.574767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.623439 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:35.678780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b4de-616f-4d55-a45a-e66239dad695 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.691553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6208c0-4da9-44d6-83a7-f8ea43107444 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.704159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:35.719056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.719849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.720226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.720591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.731476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d1b11-695b-448a-b2ab-3e7c98c47163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.732596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d1b11-695b-448a-b2ab-3e7c98c47163 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.733014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d1b11-695b-448a-b2ab-3e7c98c47163 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.733413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.733549 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d1b11-695b-448a-b2ab-3e7c98c47163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.735093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:35.752142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:35.753279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:35.754343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:35.754343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:35.754343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:35.760174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 372/751] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:35.760174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.826010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d1b11-695b-448a-b2ab-3e7c98c47163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.828436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d1b11-695b-448a-b2ab-3e7c98c47163 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.880898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:35.888059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3de6dd-bdf2-48ef-8953-6cd49e10db5f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:35.930641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:35.931847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:35.932318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:35.933062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:35.934144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=5e4604fb28234efc9f107cb1f4aa57ad) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:35.934657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:35.937029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:35.945802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 373/752] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/role_assignments?user.id=5e4604fb28234efc9f107cb1f4aa57ad&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=241e99d365564c0fa95cb712f4004785 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:35.946858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:35.968530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 380/753] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:22:35.973789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d1b11-695b-448a-b2ab-3e7c98c47163 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:36.023165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:36.129011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d1b11-695b-448a-b2ab-3e7c98c47163 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:36.171508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:36.173140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.173590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.174046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.182525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec00410-361b-4482-96cd-09f699e7cf3e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:36.188444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:36.190284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:36.209802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:36.210807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:36.211713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:36.211713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:36.211713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:36.213930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 381/754] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:36.221794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:36.245758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:36.249953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.250220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.250489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.251174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:36.251469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:36.253868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:36.279622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 374/755] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:36.280562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:36.324792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:36.331079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:36.396080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26c238-93e8-46a4-9d15-f027332eb55e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:36.455209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:36.455737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.456011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.456280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.456929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:36.457411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:36.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:36.464368 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac01228-ff75-4646-9919-b78aa24580c7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:36.483994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 382/756] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:36.489993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:36.535359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:36.535989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.536261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.536525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.552790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:36.554245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:36.577261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:36.578238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:36.579187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 375/757] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:22:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:36.584284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:36.654595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8da14-ff79-42e8-8521-4d36e9136130 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:36.685452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:36.698464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:36.702793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde6cc0-aa77-48ba-bb92-fc4bef69e97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:36.704055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde6cc0-aa77-48ba-bb92-fc4bef69e97a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.704331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde6cc0-aa77-48ba-bb92-fc4bef69e97a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.704600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde6cc0-aa77-48ba-bb92-fc4bef69e97a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.714020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde6cc0-aa77-48ba-bb92-fc4bef69e97a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:36.716161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde6cc0-aa77-48ba-bb92-fc4bef69e97a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:36.719836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69513a4-d398-4757-bf5a-34f0c374044f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0fde6cc0-aa77-48ba-bb92-fc4bef69e97a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:36.730157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:36.731174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:36.732053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:36.732053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:36.732053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:36.732053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:36.732053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:36.732053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:36.732053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:36.732053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:36.734930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 383/758] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:36.736192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc06b2-5e3e-4a03-bf33-dfb0538ee406 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.736478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc06b2-5e3e-4a03-bf33-dfb0538ee406 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.736760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc06b2-5e3e-4a03-bf33-dfb0538ee406 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.738187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 384/759] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:36.739818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc06b2-5e3e-4a03-bf33-dfb0538ee406 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:36.769628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79d881-da4a-4757-b402-350c3005278b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:36.770813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79d881-da4a-4757-b402-350c3005278b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.771278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79d881-da4a-4757-b402-350c3005278b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.771854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79d881-da4a-4757-b402-350c3005278b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.773344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79d881-da4a-4757-b402-350c3005278b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:36.773846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79d881-da4a-4757-b402-350c3005278b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:36.776236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79d881-da4a-4757-b402-350c3005278b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:36.802177 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 376/760] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:22:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:36.804121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a7bd1-7a86-4030-abde-3096a11f2c59 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.804589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a7bd1-7a86-4030-abde-3096a11f2c59 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.805064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a7bd1-7a86-4030-abde-3096a11f2c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.828852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-808a7bd1-7a86-4030-abde-3096a11f2c59 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:36.830831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-808a7bd1-7a86-4030-abde-3096a11f2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:36.857055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc06b2-5e3e-4a03-bf33-dfb0538ee406 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:36.895000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:36.895888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:36.896169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:36.896441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:36.897144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:36.897445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:36.899383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:36.919527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 385/761] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:36.920647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:36.929011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 377/762] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:36.930078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a7bd1-7a86-4030-abde-3096a11f2c59 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:36.968579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:37.031235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92236d41-047b-4506-b93e-d5d58edb5902 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:37.059054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:37.079502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:37.080273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.080657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.080937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.099114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a7bd1-7a86-4030-abde-3096a11f2c59 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:37.102567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:37.108283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:37.130966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:37.131958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:37.133247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:37.133247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:37.133247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:37.133247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 386/763] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:37.138114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:37.141681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:37.142350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.142628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.142917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.143656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=8381a7bfacbc41b39631c63b29359220) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:37.143957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:37.145404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:37.158373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 378/764] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:36 2022] GET /identity/v3/role_assignments?user.id=8381a7bfacbc41b39631c63b29359220&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:37.248581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:37.256143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53f03c-ba3f-46a7-9414-7b372ac9b416 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:37.294091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:37.294761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.295049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.295324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.296000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:37.296304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:37.297716 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:37.311710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 387/765] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:37.317402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:37.426762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:37.428294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f1f01-143c-4f07-b487-22d89ffa5162 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:37.467122 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:37.467768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.468039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.468302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.475911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:37.482701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:37.484196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:37.506543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:37.507772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:37.508826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:37.508826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:37.508826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:37.508826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:37.508826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:37.508826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:37.508826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:37.508826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:37.509252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 379/766] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:37.517519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:37.615147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6569e-fe92-4b22-8586-26a8f69b9bac None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:37.630441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f1c84-a637-4ae4-b262-bf50d65516b6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:37.673455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a007bc-1b78-454b-bac0-04acede88297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:37.674116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a007bc-1b78-454b-bac0-04acede88297 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.674394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a007bc-1b78-454b-bac0-04acede88297 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.674665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a007bc-1b78-454b-bac0-04acede88297 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.675291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a007bc-1b78-454b-bac0-04acede88297 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:37.676687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a007bc-1b78-454b-bac0-04acede88297 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:37.677704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:37.678320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.678682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.678977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.679686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:37.679989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:37.681413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e0a007bc-1b78-454b-bac0-04acede88297 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:37.698861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:37.699887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:37.700718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:37.701925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 388/767] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Aug 31 04:22:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:37.702141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a9ad-c94c-43b3-8cbf-ff0ca4970fa0 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.702814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a9ad-c94c-43b3-8cbf-ff0ca4970fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.702814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783a9ad-c94c-43b3-8cbf-ff0ca4970fa0 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.704316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 389/768] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:37.711811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783a9ad-c94c-43b3-8cbf-ff0ca4970fa0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:37.711981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 380/769] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:37.718527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:37.737236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:37.796862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:37.837511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783a9ad-c94c-43b3-8cbf-ff0ca4970fa0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:37.858136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff69ff-0dcc-40b2-9ec6-ddfa515aaca5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:37.880364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69e22a-469c-432a-9f95-570cf84fcbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:37.881862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e22a-469c-432a-9f95-570cf84fcbdf None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.882348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e22a-469c-432a-9f95-570cf84fcbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.883026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e22a-469c-432a-9f95-570cf84fcbdf None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.884252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69e22a-469c-432a-9f95-570cf84fcbdf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:37.884962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69e22a-469c-432a-9f95-570cf84fcbdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:37.887335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69e22a-469c-432a-9f95-570cf84fcbdf None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:37.894327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 390/770] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 31 04:22:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:37.895836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76b39-91b1-4021-a31c-39f87c5ccdf2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.896295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76b39-91b1-4021-a31c-39f87c5ccdf2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.896726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76b39-91b1-4021-a31c-39f87c5ccdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.899968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3731047-9925-443d-b722-d291fa3e751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:37.900696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3731047-9925-443d-b722-d291fa3e751b None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:37.901114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3731047-9925-443d-b722-d291fa3e751b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:37.901461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3731047-9925-443d-b722-d291fa3e751b None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/708ce6cfc5f64d26a418b4339fd883fb/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:37.904152 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3731047-9925-443d-b722-d291fa3e751b None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=708ce6cfc5f64d26a418b4339fd883fb, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:37.906202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3731047-9925-443d-b722-d291fa3e751b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:37.918226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f0b76b39-91b1-4021-a31c-39f87c5ccdf2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:37.919753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f0b76b39-91b1-4021-a31c-39f87c5ccdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:37.926824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 381/771] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:37 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/708ce6cfc5f64d26a418b4339fd883fb/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:22:37.928058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3731047-9925-443d-b722-d291fa3e751b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:38.064596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 391/772] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:38.085247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513a177-2742-4bbb-8793-6c512b5c8069 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:38.085585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513a177-2742-4bbb-8793-6c512b5c8069 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:38.085799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513a177-2742-4bbb-8793-6c512b5c8069 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:38.087434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 392/773] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:38.092888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11254-5a2e-48c9-bacd-0ec691ae69bc None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:38.093195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11254-5a2e-48c9-bacd-0ec691ae69bc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:38.093378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11254-5a2e-48c9-bacd-0ec691ae69bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:38.118948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3731047-9925-443d-b722-d291fa3e751b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:38.138861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b1a11254-5a2e-48c9-bacd-0ec691ae69bc None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:38.139977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b1a11254-5a2e-48c9-bacd-0ec691ae69bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:38.145409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:38.156737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:38.188289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:38.188907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:38.189179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:38.189447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:38.190083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:38.190382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:38.191816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:38.198431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 382/774] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 31 04:22:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:38.205317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:38.234149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 393/775] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:38.345639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48d90f-3664-40b4-9374-82e2d0ed26bf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:38.392960 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:38.393581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:38.393861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:38.394129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:38.394868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] RBAC: Adding query filter params (interface=public, service_id=119afb86e9344e878da7daca29fc2874, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:38.395174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:38.396547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:38.402167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a11254-5a2e-48c9-bacd-0ec691ae69bc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:38.405246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 383/776] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 31 04:22:38 2022] GET /identity/v3/endpoints?service_id=119afb86e9344e878da7daca29fc2874&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:38.410889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:38.475161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:38.564713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a4d76-8371-4fc6-9a21-48080234a652 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:38.581271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a11254-5a2e-48c9-bacd-0ec691ae69bc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:38.601599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f8282-c78a-4350-ab50-df3121e102df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:38.602129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f8282-c78a-4350-ab50-df3121e102df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:38.602401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f8282-c78a-4350-ab50-df3121e102df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:38.602671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f8282-c78a-4350-ab50-df3121e102df None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:38.603315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f8282-c78a-4350-ab50-df3121e102df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:38.604787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f8282-c78a-4350-ab50-df3121e102df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:38.618747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 384/777] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:22:38 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:38.618747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f8282-c78a-4350-ab50-df3121e102df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:38.647398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:38.648065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:38.648332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:38.648599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] PATH_INFO: `/v3/services/network` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:38.649198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:38.655422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:38.673381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:38.674632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:38.675513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 394/778] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:38 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:38.684680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:38.709442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:38.800590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f8282-c78a-4350-ab50-df3121e102df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:38.811555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:38.859547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad7fb1-7c5d-43e4-aa31-e1f13805d645 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:38.861862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531388e1-e158-4edd-b68c-154a72f40b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:38.863704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531388e1-e158-4edd-b68c-154a72f40b4b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:38.864263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531388e1-e158-4edd-b68c-154a72f40b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:38.864753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531388e1-e158-4edd-b68c-154a72f40b4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:38.879302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531388e1-e158-4edd-b68c-154a72f40b4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-531388e1-e158-4edd-b68c-154a72f40b4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:38.888275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:38.889234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:38.890117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:38.893598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 385/779] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:38.898597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531388e1-e158-4edd-b68c-154a72f40b4b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:38.905204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:38.905830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:38.906113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:38.906379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:38.907070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:38.907368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:38.908750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:38.917033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 395/780] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:38.928803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:39.100532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:39.176233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c7769-53c8-4c74-b089-940cb65084f9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:39.216564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531388e1-e158-4edd-b68c-154a72f40b4b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:39.254313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:39.255338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.255815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.256279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.257174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:39.257675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:39.259270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:39.269180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 386/781] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:39.272398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:39.273274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.273764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.274910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:39.275746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.278734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] RBAC: Adding query filter params (type=network) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:39.280241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:39.281841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:39.296113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 396/782] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:39.302071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:39.331943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:39.397785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975b353-6bd1-43be-b3f2-053349b5fc37 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:39.453083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf52088-799a-4040-8992-207a6ba3cb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:39.454598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52088-799a-4040-8992-207a6ba3cb10 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.455363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52088-799a-4040-8992-207a6ba3cb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.455832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52088-799a-4040-8992-207a6ba3cb10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.468796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf52088-799a-4040-8992-207a6ba3cb10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:39.470741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf52088-799a-4040-8992-207a6ba3cb10 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:39.471216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528a546-dfe2-423a-9500-2997fda7877b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-daf52088-799a-4040-8992-207a6ba3cb10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:39.486489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:39.487467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:39.488322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:39.488322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:39.488322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:39.490432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 387/783] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:39.492012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910774d-d93f-4ebb-9d81-b8608ab425f3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.492498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910774d-d93f-4ebb-9d81-b8608ab425f3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.493006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910774d-d93f-4ebb-9d81-b8608ab425f3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.494671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 388/784] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:39.495821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910774d-d93f-4ebb-9d81-b8608ab425f3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:39.510667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897688fa-b4e3-4e6f-9c3b-bd7b40f51bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:39.511477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897688fa-b4e3-4e6f-9c3b-bd7b40f51bfe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.511761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897688fa-b4e3-4e6f-9c3b-bd7b40f51bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.512050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897688fa-b4e3-4e6f-9c3b-bd7b40f51bfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.512861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897688fa-b4e3-4e6f-9c3b-bd7b40f51bfe None admin] RBAC: Adding query filter params (interface=public, service_id=af545604a79a4608852dbe010a8b3dc2, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:39.513069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897688fa-b4e3-4e6f-9c3b-bd7b40f51bfe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:39.514457 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897688fa-b4e3-4e6f-9c3b-bd7b40f51bfe None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:39.519269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:39.522877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 397/785] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 31 04:22:39 2022] GET /identity/v3/endpoints?service_id=af545604a79a4608852dbe010a8b3dc2&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:39.527358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68371b-77d4-43d4-9d82-baccd3f23a34 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.527358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68371b-77d4-43d4-9d82-baccd3f23a34 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.527358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68371b-77d4-43d4-9d82-baccd3f23a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.544301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fa68371b-77d4-43d4-9d82-baccd3f23a34 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:39.545653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fa68371b-77d4-43d4-9d82-baccd3f23a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:39.588215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:39.611630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910774d-d93f-4ebb-9d81-b8608ab425f3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:39.618978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 398/786] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:39.620266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68371b-77d4-43d4-9d82-baccd3f23a34 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:39.648877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:39.649727 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.650163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.650556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.651419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:39.651982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:39.653530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:39.666610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 389/787] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:39.672982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:39.904484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68371b-77d4-43d4-9d82-baccd3f23a34 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:39.936075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:39.943327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e389e-3fe0-46a2-96e3-450a430534bc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:39.948671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302984-6ebf-4673-b451-7e8ebadb7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:39.949484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302984-6ebf-4673-b451-7e8ebadb7765 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.949905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302984-6ebf-4673-b451-7e8ebadb7765 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.950314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302984-6ebf-4673-b451-7e8ebadb7765 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.951608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51302984-6ebf-4673-b451-7e8ebadb7765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:39.953164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51302984-6ebf-4673-b451-7e8ebadb7765 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:39.961100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 399/788] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:22:39 2022] GET /identity/v3/services => generated 772 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:39.962502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302984-6ebf-4673-b451-7e8ebadb7765 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:39.982337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:39.983163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:39.983538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:39.984249 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:39.993976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:39.995834 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:39.999149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:40.009508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:40.010562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:40.011759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 390/789] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:40.016807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:40.090872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302984-6ebf-4673-b451-7e8ebadb7765 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:40.128930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc997cd-e1da-4933-93c4-676c58655022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:40.130006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc997cd-e1da-4933-93c4-676c58655022 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.130483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc997cd-e1da-4933-93c4-676c58655022 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.130981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc997cd-e1da-4933-93c4-676c58655022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.143789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc997cd-e1da-4933-93c4-676c58655022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0fc997cd-e1da-4933-93c4-676c58655022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:40.149237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:40.150253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:40.151114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:40.154322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 400/790] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:40.160358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc997cd-e1da-4933-93c4-676c58655022 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:40.228943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:40.250903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617be60-954a-41d0-a4df-00910299a6d3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:40.284461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc997cd-e1da-4933-93c4-676c58655022 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:40.288369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd9187-684a-4374-baca-e1910b173d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:40.288808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd9187-684a-4374-baca-e1910b173d13 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.289086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd9187-684a-4374-baca-e1910b173d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.289354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd9187-684a-4374-baca-e1910b173d13 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.290091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd9187-684a-4374-baca-e1910b173d13 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:40.290402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd9187-684a-4374-baca-e1910b173d13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:40.291863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd9187-684a-4374-baca-e1910b173d13 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:40.310563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 391/791] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 31 04:22:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:40.311875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb71648-dc16-4738-b60a-ac09be273df7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.312156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb71648-dc16-4738-b60a-ac09be273df7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.312426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb71648-dc16-4738-b60a-ac09be273df7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.317621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 392/792] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:40.317621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb71648-dc16-4738-b60a-ac09be273df7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:40.330897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58f5b1-fea8-4eca-b5f7-91b694d44643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:40.331577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58f5b1-fea8-4eca-b5f7-91b694d44643 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.331858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58f5b1-fea8-4eca-b5f7-91b694d44643 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.332134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58f5b1-fea8-4eca-b5f7-91b694d44643 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.332816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58f5b1-fea8-4eca-b5f7-91b694d44643 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:40.333538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58f5b1-fea8-4eca-b5f7-91b694d44643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:40.335361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58f5b1-fea8-4eca-b5f7-91b694d44643 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:40.347787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 401/793] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:40.349272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1604d4e-4fd2-4edc-8639-333f9609a5a0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.349572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1604d4e-4fd2-4edc-8639-333f9609a5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.350165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1604d4e-4fd2-4edc-8639-333f9609a5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.373241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b1604d4e-4fd2-4edc-8639-333f9609a5a0 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:40.374781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b1604d4e-4fd2-4edc-8639-333f9609a5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:40.392227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:40.445969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb71648-dc16-4738-b60a-ac09be273df7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:40.470747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 402/794] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:40.470747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1604d4e-4fd2-4edc-8639-333f9609a5a0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:40.513211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:40.513853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.514128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.514394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.525473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:40.527046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:40.542998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:40.544172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:40.545034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:40.545034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:40.545034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:40.545034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 393/795] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:40.551777 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:40.577307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:40.645553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1604d4e-4fd2-4edc-8639-333f9609a5a0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:40.684500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:40.685139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.685418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.685685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.702285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:40.703710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:40.730264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:40.731273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:40.732146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:40.732146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:40.732146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:40.732146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 403/796] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:40.738593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:40.759795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507995ec-0672-4090-9303-171aa26664c4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:40.774218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:40.807533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:40.808598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.809076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.809537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.810361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:40.810909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:40.812490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:40.825721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 394/797] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:40.826867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:40.887237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:40.894373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca7e7c-08d8-4fcb-bded-f5b0a0d4931d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:40.930400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:40.931330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4d235-b221-4410-9493-96f068e75a90 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:40.931606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.932070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.932557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.933454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:40.934031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:40.935473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:40.952871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 404/798] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:40.952871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:40.974737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49dc09-b62e-4bc8-b143-d4641acea1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:40.975371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49dc09-b62e-4bc8-b143-d4641acea1da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.975721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49dc09-b62e-4bc8-b143-d4641acea1da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.975995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49dc09-b62e-4bc8-b143-d4641acea1da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:40.986624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49dc09-b62e-4bc8-b143-d4641acea1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9c49dc09-b62e-4bc8-b143-d4641acea1da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:40.991440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:40.992418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:40.993226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:40.997571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 395/799] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:40.998656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afda16-faba-4822-bc1e-0c7cca2473fc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:40.999047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afda16-faba-4822-bc1e-0c7cca2473fc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:40.999296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afda16-faba-4822-bc1e-0c7cca2473fc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.006243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 396/800] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:41.006243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afda16-faba-4822-bc1e-0c7cca2473fc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.038666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:41.076764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbda3ae-3871-458d-aa8c-741a99f2dced None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.114540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afda16-faba-4822-bc1e-0c7cca2473fc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.117253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.117898 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.118180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.118459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.125790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.127361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:41.139297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:41.140211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:41.141117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 405/801] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:41.141596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.158165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93d264-5a4b-413f-879a-923a6e366729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.159134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93d264-5a4b-413f-879a-923a6e366729 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.159590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93d264-5a4b-413f-879a-923a6e366729 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.160054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93d264-5a4b-413f-879a-923a6e366729 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.163721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:41.169220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93d264-5a4b-413f-879a-923a6e366729 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.170687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93d264-5a4b-413f-879a-923a6e366729 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6b93d264-5a4b-413f-879a-923a6e366729 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:41.182578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:41.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:41.184708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 397/802] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:41.185752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f61b-fda0-488a-8ec6-377a5f2ac601 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.185752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f61b-fda0-488a-8ec6-377a5f2ac601 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.185903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f61b-fda0-488a-8ec6-377a5f2ac601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.204254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-10f5f61b-fda0-488a-8ec6-377a5f2ac601 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:41.205761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10f5f61b-fda0-488a-8ec6-377a5f2ac601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:41.259728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e4b50-edc7-46ae-8ea0-d85e80907ad3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.282803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 398/803] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:41.282803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f61b-fda0-488a-8ec6-377a5f2ac601 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.310742 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:41.314973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.315603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.315885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.316161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.316839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:41.317142 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.318941 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.328125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 406/804] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:41.329042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.415153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:41.415357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5f61b-fda0-488a-8ec6-377a5f2ac601 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.438757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be021c-bc0e-41a9-8397-d50343d3b0e5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.457619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.458242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.458441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.458798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.459743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:41.460050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.462396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.478206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 399/805] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:41.479446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.485629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.486509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.487037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.487501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.488492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:41.489519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.491681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.515625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 407/806] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:41.517111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.576161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:41.598260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30759882-3f67-4aa5-81cf-ea05f712e455 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.634425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.635105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.635384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.635660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.635782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2c21c-8d9d-465a-a9b4-74bbee31b7e5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.647425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.648884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:41.663861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:41.665004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:41.665906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:41.665906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:41.665906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:41.665906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 400/807] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:41.665906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.677576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f94b16-6700-4060-be8c-0c6c83fb4896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.678252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f94b16-6700-4060-be8c-0c6c83fb4896 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.678563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f94b16-6700-4060-be8c-0c6c83fb4896 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.678871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f94b16-6700-4060-be8c-0c6c83fb4896 None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/5e4604fb28234efc9f107cb1f4aa57ad/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.680754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f94b16-6700-4060-be8c-0c6c83fb4896 None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=5e4604fb28234efc9f107cb1f4aa57ad, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.682204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f94b16-6700-4060-be8c-0c6c83fb4896 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.685902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:41.699581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 408/808] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:41 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/5e4604fb28234efc9f107cb1f4aa57ad/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:22:41.700775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f94b16-6700-4060-be8c-0c6c83fb4896 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.812906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f94b16-6700-4060-be8c-0c6c83fb4896 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.822006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9942d3-f3d4-49f0-a2b6-bd6e688eeeb4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:41.852839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.853502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.853782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.854057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.854666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.856632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.856925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:41.864514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:41.865464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:41.865920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:41.866366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:41.869411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:41.870449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 04:22:41.871483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:41.875146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 409/809] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:41.876075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.884622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:41.886640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:41.902665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:41.903756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:41.904614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:41.904614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:41.904614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:41.904614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 401/810] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:41.904614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:41.958820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:42.003495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc095fe-3ee6-4e0f-809a-76d05bbcec8f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.040255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9595-9cad-45e8-9361-b1ebe44afd19 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.046237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.046893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.047176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.047442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.048090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:42.048388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.050232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.063692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 410/811] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:42.064477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.083689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ce2ce-4542-4fd1-87d4-e32290f2eeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.084301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce2ce-4542-4fd1-87d4-e32290f2eeaf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.084565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce2ce-4542-4fd1-87d4-e32290f2eeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.084837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce2ce-4542-4fd1-87d4-e32290f2eeaf None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.085494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ce2ce-4542-4fd1-87d4-e32290f2eeaf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:42.086076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ce2ce-4542-4fd1-87d4-e32290f2eeaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.087183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ce2ce-4542-4fd1-87d4-e32290f2eeaf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.092085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 402/812] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 31 04:22:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:42.097624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445307cf-bee6-4efe-8df3-59a12b3a5818 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.097992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445307cf-bee6-4efe-8df3-59a12b3a5818 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.098177 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445307cf-bee6-4efe-8df3-59a12b3a5818 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.099423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 403/813] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:42.103168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445307cf-bee6-4efe-8df3-59a12b3a5818 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.152800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:42.175571 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dc585-ff5e-4554-ad98-38dd2917e655 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.195474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:42.211092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445307cf-bee6-4efe-8df3-59a12b3a5818 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.213102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.213633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.213921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.214178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.215601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:42.215601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.216665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.233073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 411/814] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:42.233073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.256589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f74299-845f-4272-ae04-0614f8c36152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.257325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f74299-845f-4272-ae04-0614f8c36152 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.257479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f74299-845f-4272-ae04-0614f8c36152 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.257769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f74299-845f-4272-ae04-0614f8c36152 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.266121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f74299-845f-4272-ae04-0614f8c36152 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.267588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f74299-845f-4272-ae04-0614f8c36152 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b9f74299-845f-4272-ae04-0614f8c36152 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:42.279210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:42.280156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:42.281063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 404/815] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:22:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:42.282136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762b0c-b0e8-497d-a8e0-d4a8ec97e946 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.282136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762b0c-b0e8-497d-a8e0-d4a8ec97e946 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.282136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762b0c-b0e8-497d-a8e0-d4a8ec97e946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.300991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-49762b0c-b0e8-497d-a8e0-d4a8ec97e946 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:42.302154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-49762b0c-b0e8-497d-a8e0-d4a8ec97e946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:42.338995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d21e1-a791-47e0-9650-ad6f20d3256e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.374692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 405/816] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:42.375570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49762b0c-b0e8-497d-a8e0-d4a8ec97e946 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.384685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.385511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.385954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.386476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.387425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:42.388073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.389753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.395390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 412/817] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Aug 31 04:22:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:42.396527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.422587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:42.451875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:42.492908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49762b0c-b0e8-497d-a8e0-d4a8ec97e946 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.524021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339f059-6f38-4f86-9da3-239ab6d2c8cc None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.529578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.530418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.530892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.531305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.538915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.540675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:42.553433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:42.554305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:42.555297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:42.555297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:42.555297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:42.555297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:42.555297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:42.555297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:42.555297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:42.555297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:42.562129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 406/818] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:42.562129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.569714 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.570964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.571481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.572018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.572988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:42.573483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.575404 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.599466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 413/819] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:22:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:42.600712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.671989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:42.676482 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e91c4-0bdb-477c-8a15-5d3b776253b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.708727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecba48b-3103-46c5-8527-8d58ba9a3552 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.710258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:42.710531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940d6b7-08a7-4d51-a2bd-8cab13961ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.711574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d6b7-08a7-4d51-a2bd-8cab13961ddc None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.711882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d6b7-08a7-4d51-a2bd-8cab13961ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.712993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d6b7-08a7-4d51-a2bd-8cab13961ddc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.712993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940d6b7-08a7-4d51-a2bd-8cab13961ddc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.714134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940d6b7-08a7-4d51-a2bd-8cab13961ddc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.746757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 407/820] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:22:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:42.746757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940d6b7-08a7-4d51-a2bd-8cab13961ddc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.761119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.762029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.762343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.762778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.763691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:42.764002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.765651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.780439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 414/821] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:42.781438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:42.893812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940d6b7-08a7-4d51-a2bd-8cab13961ddc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.915953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb28e58-cdf8-4e37-b652-ad937edb6575 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:42.933215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ff369-4565-4212-8de4-8b5ab7aae0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.933809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff369-4565-4212-8de4-8b5ab7aae0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.934089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff369-4565-4212-8de4-8b5ab7aae0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.934359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff369-4565-4212-8de4-8b5ab7aae0b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.957118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:42.958569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:42.959314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:42.960156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] PATH_INFO: `/v3/services/network` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:42.960807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:42.962177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:42.964501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:42.975552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:42.976547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 04:22:42.977389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:42.981773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 415/822] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:42 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:42.983201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:43.012225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ff369-4565-4212-8de4-8b5ab7aae0b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:43.013692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ff369-4565-4212-8de4-8b5ab7aae0b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:43.015287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:43.111977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ceca4-d07c-4a53-bfe9-6bf72bcf7b23 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:43.116088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 408/823] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:22:43.116961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ff369-4565-4212-8de4-8b5ab7aae0b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:43.149531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de5f5-44bb-43a6-9f7d-d547513e0f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:43.150212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de5f5-44bb-43a6-9f7d-d547513e0f9d None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.150489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de5f5-44bb-43a6-9f7d-d547513e0f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.150784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de5f5-44bb-43a6-9f7d-d547513e0f9d None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/8381a7bfacbc41b39631c63b29359220/roles/efe57f5de58341b6898d0d4f45a9ffc0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.167098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503de5f5-44bb-43a6-9f7d-d547513e0f9d None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=8381a7bfacbc41b39631c63b29359220, role_id=efe57f5de58341b6898d0d4f45a9ffc0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:43.168871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503de5f5-44bb-43a6-9f7d-d547513e0f9d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:43.179014 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:43.190042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 416/824] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:42 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/8381a7bfacbc41b39631c63b29359220/roles/efe57f5de58341b6898d0d4f45a9ffc0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:22:43.191020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de5f5-44bb-43a6-9f7d-d547513e0f9d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:43.267213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ff369-4565-4212-8de4-8b5ab7aae0b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:43.280461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:43.335603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de5f5-44bb-43a6-9f7d-d547513e0f9d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:43.349312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:43.349948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.350272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.350540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.351353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] RBAC: Adding query filter params (name=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:43.351657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:43.353056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:43.362337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 409/825] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:43.368327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:43.415686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:43.416248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.416519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.416783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.430506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:43.431783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:43.450540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:43.451586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:43.452462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:43.452462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:43.452462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:43.452462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 417/826] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:43.457477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:43.517244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4c89-8ed8-413f-93be-999e6120456b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:43.544464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:43.571448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:43.572060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.572334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.572600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.573248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] RBAC: Adding query filter params (type=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:43.573539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:43.574973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:43.587751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc383-01ee-4de0-bfee-f4f89ebadf72 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:43.590081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 410/827] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:43.597249 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:43.615764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:43.629765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:43.630752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.631240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.631973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.632846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:43.633362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:43.635021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:43.649200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 418/828] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:43.654639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:43.767070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d91e16-f11f-44dd-95ff-9f04996fa4ff None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:43.801668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c9176-f259-4dc0-a611-ea193443044a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:43.806183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e730a1d-3f37-4066-8eaf-8f989ebf8298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:43.806885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730a1d-3f37-4066-8eaf-8f989ebf8298 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.807169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730a1d-3f37-4066-8eaf-8f989ebf8298 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.807492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730a1d-3f37-4066-8eaf-8f989ebf8298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.814950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e730a1d-3f37-4066-8eaf-8f989ebf8298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:43.816771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e730a1d-3f37-4066-8eaf-8f989ebf8298 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-1e730a1d-3f37-4066-8eaf-8f989ebf8298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:43.831569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:43.832489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:43.833367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 419/829] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:43.834332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b095-1156-421a-ba5e-99a70683fea9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.834428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b095-1156-421a-ba5e-99a70683fea9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.834661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b095-1156-421a-ba5e-99a70683fea9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.838793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 420/830] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:43.840399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15b095-1156-421a-ba5e-99a70683fea9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:43.853759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42111b3d-4891-46e9-aab1-fa2f442be71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:43.854764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111b3d-4891-46e9-aab1-fa2f442be71a None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.856012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111b3d-4891-46e9-aab1-fa2f442be71a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.856612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111b3d-4891-46e9-aab1-fa2f442be71a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.857581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42111b3d-4891-46e9-aab1-fa2f442be71a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:43.860003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42111b3d-4891-46e9-aab1-fa2f442be71a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:43.889630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 411/831] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:22:43 2022] POST /identity/v3/endpoints => generated 337 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:22:43.894861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21927997-8c34-42f1-971b-826d870305f1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:43.894861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21927997-8c34-42f1-971b-826d870305f1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:43.894861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21927997-8c34-42f1-971b-826d870305f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:43.920558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:43.929687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-21927997-8c34-42f1-971b-826d870305f1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:43.930928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-21927997-8c34-42f1-971b-826d870305f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:43.997489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:44.047905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15b095-1156-421a-ba5e-99a70683fea9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:44.085668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 412/832] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:44.102427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:44.103087 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:44.103363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:44.103634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:44.104333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:44.104639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:44.106185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:44.122830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 421/833] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:44.124745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21927997-8c34-42f1-971b-826d870305f1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:44.223508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21927997-8c34-42f1-971b-826d870305f1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:44.257455 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:44.258065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:44.258342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:44.258612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:44.259377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=708ce6cfc5f64d26a418b4339fd883fb) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:44.259737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:44.261179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:44.270797 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 413/834] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:44 2022] GET /identity/v3/role_assignments?user.id=708ce6cfc5f64d26a418b4339fd883fb&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=241e99d365564c0fa95cb712f4004785 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:44.403130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:44.504666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:44.507748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8435-061f-4292-9fff-e9be1757463c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:44.543886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cabb30-cdbe-4fe9-87e2-5aee200abce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:44.544521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cabb30-cdbe-4fe9-87e2-5aee200abce8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:44.544791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cabb30-cdbe-4fe9-87e2-5aee200abce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:44.545054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cabb30-cdbe-4fe9-87e2-5aee200abce8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:44.555329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cabb30-cdbe-4fe9-87e2-5aee200abce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f2cabb30-cdbe-4fe9-87e2-5aee200abce8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:44.560126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:44.561065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:44.561903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 422/835] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:44.567532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:44.603460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:44.672717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca40fdd-eda1-4f2d-ac83-e64b08c8dce9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:44.707121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:44.707789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:44.708070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:44.708345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:44.709018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:44.709317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:44.710810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:44.719761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 414/836] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:22:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:44.725294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cabb30-cdbe-4fe9-87e2-5aee200abce8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:44.825116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cabb30-cdbe-4fe9-87e2-5aee200abce8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:44.858885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510503ec-d065-4bfc-9d96-ebadac6b3225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:44.859523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510503ec-d065-4bfc-9d96-ebadac6b3225 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:44.859796 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510503ec-d065-4bfc-9d96-ebadac6b3225 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:44.860060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510503ec-d065-4bfc-9d96-ebadac6b3225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:44.870846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510503ec-d065-4bfc-9d96-ebadac6b3225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:44.872434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510503ec-d065-4bfc-9d96-ebadac6b3225 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-510503ec-d065-4bfc-9d96-ebadac6b3225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:44.886593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:44.887579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:44.888439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:44.888439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:44.888439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:44.888439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 423/837] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:44.893848 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:44.932268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d814a-6698-48e3-abc2-87c2b6f31ed1 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:44.932443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d814a-6698-48e3-abc2-87c2b6f31ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:44.932720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d814a-6698-48e3-abc2-87c2b6f31ed1 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:44.934294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 424/838] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:22:44.940033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3f35b-a73e-43c3-8100-47d84f693d29 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:44.940313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3f35b-a73e-43c3-8100-47d84f693d29 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:44.940585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3f35b-a73e-43c3-8100-47d84f693d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:44.964045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-91f3f35b-a73e-43c3-8100-47d84f693d29 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:44.965718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-91f3f35b-a73e-43c3-8100-47d84f693d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:44.975718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.003535 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.012386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc127c-1f81-49bc-bfdb-5b4d47938ffe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.048892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.049768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.050262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.050685 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.051583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:45.052028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:45.053584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:45.062755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 425/839] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:45.077477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 415/840] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:45.083237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3f35b-a73e-43c3-8100-47d84f693d29 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:45.186736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3f35b-a73e-43c3-8100-47d84f693d29 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.223076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.224630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.225289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.225966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.234123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:45.235884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:45.248546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:45.249487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 04:22:45.250408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:45.252926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 426/841] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:22:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:45.258885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:45.335543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:45.338890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.375123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53fca6-de99-4380-8612-e2d23c1213c1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.412783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.413402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.413675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.414037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.414693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:45.415043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:45.416486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:45.441013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 416/842] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Aug 31 04:22:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:45.443581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:45.462110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.470934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940b4dc-9ab1-43ec-861f-3ca1def3a72b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.510988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756faf8-331b-4979-a1fe-ea62d1b02f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.512302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756faf8-331b-4979-a1fe-ea62d1b02f04 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.512591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756faf8-331b-4979-a1fe-ea62d1b02f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.512869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756faf8-331b-4979-a1fe-ea62d1b02f04 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.513498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756faf8-331b-4979-a1fe-ea62d1b02f04 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:45.514914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756faf8-331b-4979-a1fe-ea62d1b02f04 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-1756faf8-331b-4979-a1fe-ea62d1b02f04 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:45.523738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:45.524698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:45.525539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 427/843] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:45.526306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8cc77-93a7-4b2a-8f50-31fa4e654de1 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.526306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8cc77-93a7-4b2a-8f50-31fa4e654de1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.526800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8cc77-93a7-4b2a-8f50-31fa4e654de1 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.532317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 428/844] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:45.533097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8cc77-93a7-4b2a-8f50-31fa4e654de1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:45.558817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e10da-a710-4c68-9a26-b12965295bff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.595358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6aa8-07aa-437e-b0e1-8b549ecf2ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.596013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6aa8-07aa-437e-b0e1-8b549ecf2ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.596309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6aa8-07aa-437e-b0e1-8b549ecf2ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.596633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6aa8-07aa-437e-b0e1-8b549ecf2ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.607601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.647146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8cc77-93a7-4b2a-8f50-31fa4e654de1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.674579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca6aa8-07aa-437e-b0e1-8b549ecf2ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:45.676333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca6aa8-07aa-437e-b0e1-8b549ecf2ea7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:45.690676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.696490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501f579-20f7-4004-8a10-c6f540fd5749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.697324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f579-20f7-4004-8a10-c6f540fd5749 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.697778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f579-20f7-4004-8a10-c6f540fd5749 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.698251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f579-20f7-4004-8a10-c6f540fd5749 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.700311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501f579-20f7-4004-8a10-c6f540fd5749 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:45.700799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501f579-20f7-4004-8a10-c6f540fd5749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:45.702886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501f579-20f7-4004-8a10-c6f540fd5749 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:45.709262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 429/845] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:45.710786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b057ea-c8e5-4d18-be7e-8bc79138abe9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.711234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b057ea-c8e5-4d18-be7e-8bc79138abe9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.712420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b057ea-c8e5-4d18-be7e-8bc79138abe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.732396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f3b057ea-c8e5-4d18-be7e-8bc79138abe9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:45.733563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f3b057ea-c8e5-4d18-be7e-8bc79138abe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:45.783503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 417/846] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:22:45.784370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6aa8-07aa-437e-b0e1-8b549ecf2ea7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:45.814769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 430/847] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:45.814769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b057ea-c8e5-4d18-be7e-8bc79138abe9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:45.891902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.902509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6aa8-07aa-437e-b0e1-8b549ecf2ea7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.919504 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.943550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.944445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.944858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.945260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.946105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:45.946451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b057ea-c8e5-4d18-be7e-8bc79138abe9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.946764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:45.948370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:45.953776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 418/848] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:45.996578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.997669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.998085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.998491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.010705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:46.012894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:46.030202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:46.031426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:46.032524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:46.032524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:46.032524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:46.034315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 431/849] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:46.039927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.108082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.169490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42193e82-a7b1-4c14-ae3d-1f667c164137 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:46.208009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:46.208628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.208911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.209185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.209843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:46.210146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:46.211754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:46.225885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 419/850] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:46.232139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.238091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f218e2b-f2e1-43f1-ab9c-e267a57545c9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:46.254062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:46.272808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:46.281192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e908b-0002-4573-a891-2d6cb0f61594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:46.281959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e908b-0002-4573-a891-2d6cb0f61594 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.282916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e908b-0002-4573-a891-2d6cb0f61594 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.283274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e908b-0002-4573-a891-2d6cb0f61594 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.297123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e908b-0002-4573-a891-2d6cb0f61594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7b7e908b-0002-4573-a891-2d6cb0f61594 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:46.303965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:46.304999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:46.305827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 432/851] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:46.311891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e908b-0002-4573-a891-2d6cb0f61594 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.360817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4202d-1966-439a-bbe4-34c3d6d89bd7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:46.396486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592a175-40a7-44b6-a4b9-59f0edc1f982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:46.397119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592a175-40a7-44b6-a4b9-59f0edc1f982 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.397399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592a175-40a7-44b6-a4b9-59f0edc1f982 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.397668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592a175-40a7-44b6-a4b9-59f0edc1f982 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.405284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592a175-40a7-44b6-a4b9-59f0edc1f982 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:46.406752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592a175-40a7-44b6-a4b9-59f0edc1f982 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b592a175-40a7-44b6-a4b9-59f0edc1f982 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:46.420490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:46.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:46.422917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 420/852] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:46.423403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a271ff4-30aa-496b-bd54-97aae0a227f9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.423639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a271ff4-30aa-496b-bd54-97aae0a227f9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.423809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a271ff4-30aa-496b-bd54-97aae0a227f9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.425185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 421/853] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:46.426329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e908b-0002-4573-a891-2d6cb0f61594 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:46.429712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a271ff4-30aa-496b-bd54-97aae0a227f9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.469350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a77d2-ea58-4b10-a496-fc3d39e06e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:46.469886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a77d2-ea58-4b10-a496-fc3d39e06e50 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.470165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a77d2-ea58-4b10-a496-fc3d39e06e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.470434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a77d2-ea58-4b10-a496-fc3d39e06e50 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.471128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a77d2-ea58-4b10-a496-fc3d39e06e50 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:46.471432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a77d2-ea58-4b10-a496-fc3d39e06e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:46.472808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a77d2-ea58-4b10-a496-fc3d39e06e50 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:46.486326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 433/854] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:46.486326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93aea6-d109-4e5a-b5cc-473e16ba4c08 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.486326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93aea6-d109-4e5a-b5cc-473e16ba4c08 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.486326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93aea6-d109-4e5a-b5cc-473e16ba4c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.502946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-eb93aea6-d109-4e5a-b5cc-473e16ba4c08 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:46.504142 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-eb93aea6-d109-4e5a-b5cc-473e16ba4c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:46.531875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:46.534349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:46.541470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a271ff4-30aa-496b-bd54-97aae0a227f9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:46.578924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 434/855] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:46.579680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:46.579820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93aea6-d109-4e5a-b5cc-473e16ba4c08 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.580859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.581334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.581902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.582821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:46.583335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:46.584983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:46.600139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 422/856] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:46.606019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.709838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93aea6-d109-4e5a-b5cc-473e16ba4c08 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:46.720674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315da060-a111-4a85-aafa-35b850050c8e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:46.752632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:46.753273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.753554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.753819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.759129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:46.759975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.760449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.761007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.762649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=5e4604fb28234efc9f107cb1f4aa57ad) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:46.763259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:46.764560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:46.765248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:46.765989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:46.772167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 423/857] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/role_assignments?user.id=5e4604fb28234efc9f107cb1f4aa57ad&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=241e99d365564c0fa95cb712f4004785 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:46.788001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:46.789038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:46.789943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:46.789943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:46.789943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:46.792366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 435/858] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:46.798970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.827558 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:46.865841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.887089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:46.918542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f16a1-e482-4d43-a6dc-7bf4f2380b7f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:46.962540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:46.963295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:46.963576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:46.963846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:46.964519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:46.964826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:46.966308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:46.978542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 424/859] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:46.984472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:46.987828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedcb57-f06d-4c84-b855-d5493b3eaa4f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:47.030143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:47.030858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.031138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.031412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.032004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:47.033379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:47.042292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:47.043479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:47.044338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:47.048426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 436/860] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:47.051080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:47.083705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:47.106908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950d73-9c20-466e-b96e-82e9d31264ea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:47.146520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:47.147205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.147484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.147757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.157488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:47.157968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:47.159904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:47.179043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:47.180125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:47.181035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:47.182802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 425/861] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:22:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:47.187663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:47.188037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129a98b-02f7-44ad-a0bd-bcdae03c7cbf None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:47.224374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:47.225121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.225460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.225801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.226505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:47.226905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:47.228354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:47.233999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 437/862] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:47.242325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:47.305595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e29f92-1241-4e20-bae6-a8b8fcdbb503 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:47.343289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:47.343904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.344180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.344449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.345196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:47.345496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:47.346960 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:47.347855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0470df-be6f-4d49-a5eb-3f0c04dcc9f0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:47.358639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:47.370348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 426/863] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 31 04:22:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:47.372047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:47.384074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:47.384795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.385079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.385351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.386438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:47.386761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:47.388344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:47.398338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 438/864] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:47.483244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc531a9-faaa-4498-ac75-5cb0c2e189a5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:47.521000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb5bd-e6b8-474d-b1db-6f765ba35fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:47.521546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb5bd-e6b8-474d-b1db-6f765ba35fce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.521828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb5bd-e6b8-474d-b1db-6f765ba35fce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.522096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb5bd-e6b8-474d-b1db-6f765ba35fce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.591671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:47.595198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cb5bd-e6b8-474d-b1db-6f765ba35fce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:47.596954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cb5bd-e6b8-474d-b1db-6f765ba35fce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:47.700977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 427/865] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 04:22:47.702364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:47.713928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:47.805982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91454e-5441-49e6-bd9f-f1f8ad8592f1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:47.840556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20751b-61f9-411c-8f4c-972e600f33a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:47.841205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20751b-61f9-411c-8f4c-972e600f33a1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.841487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20751b-61f9-411c-8f4c-972e600f33a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.841755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20751b-61f9-411c-8f4c-972e600f33a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.853791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20751b-61f9-411c-8f4c-972e600f33a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:47.855615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20751b-61f9-411c-8f4c-972e600f33a1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-1e20751b-61f9-411c-8f4c-972e600f33a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:47.870645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:47.871670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:47.872556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:47.872556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:47.872556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:47.873785 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 439/866] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:47.879692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb5bd-e6b8-474d-b1db-6f765ba35fce None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:47.931292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e317-575f-4ef2-ae3f-0dc1f645f4df None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.931627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e317-575f-4ef2-ae3f-0dc1f645f4df None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.931845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e317-575f-4ef2-ae3f-0dc1f645f4df None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.933492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 440/867] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:47.940048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd943d9-36b3-4bac-a256-f33bf69a281b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:47.940332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd943d9-36b3-4bac-a256-f33bf69a281b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:47.940608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd943d9-36b3-4bac-a256-f33bf69a281b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:47.960574 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3fd943d9-36b3-4bac-a256-f33bf69a281b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:47.961773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3fd943d9-36b3-4bac-a256-f33bf69a281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:47.981881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:47.986545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb5bd-e6b8-474d-b1db-6f765ba35fce None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:48.030983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:48.031764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:48.032261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:48.032641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:48.033692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:48.034219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:48.035895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:48.037122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:48.047105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 441/868] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:22:48.052775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 428/869] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:48.054240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd943d9-36b3-4bac-a256-f33bf69a281b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:48.154096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd943d9-36b3-4bac-a256-f33bf69a281b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:48.187558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:48.188245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:48.188454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:48.188715 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:48.196072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:48.197592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:48.208787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:48.209650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:48.210523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:48.210523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:48.210523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:48.210523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:48.210523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:48.210523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:48.210523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:48.210523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:48.214690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 442/870] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:48.216083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:48.314937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c9f68-955c-49ff-a5a2-c686b768300b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:48.320219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:48.362566 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:48.363119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:48.363395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:48.363663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:48.364716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:48.364716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:48.366072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:48.379799 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 429/871] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:48.380969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:48.385677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:48.407604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:48.444149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0999dc-81fc-4511-b781-794c83cccedb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:48.481805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06e92-ab06-4d4d-968d-c9e62f27b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:48.482876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06e92-ab06-4d4d-968d-c9e62f27b480 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:48.483381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06e92-ab06-4d4d-968d-c9e62f27b480 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:48.483852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06e92-ab06-4d4d-968d-c9e62f27b480 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:48.484718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06e92-ab06-4d4d-968d-c9e62f27b480 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:48.486298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06e92-ab06-4d4d-968d-c9e62f27b480 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:48.507555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 443/872] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:22:48 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:22:48.518852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b50cb-de72-4797-8189-80ec859bc7cc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:48.566235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27df12-6dc6-4a75-92bd-a616205475ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:48.567183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27df12-6dc6-4a75-92bd-a616205475ae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:48.567616 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27df12-6dc6-4a75-92bd-a616205475ae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:48.568420 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27df12-6dc6-4a75-92bd-a616205475ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:48.569342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27df12-6dc6-4a75-92bd-a616205475ae None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=8381a7bfacbc41b39631c63b29359220) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:48.569842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27df12-6dc6-4a75-92bd-a616205475ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:48.571539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27df12-6dc6-4a75-92bd-a616205475ae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:48.586829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 430/873] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:48 2022] GET /identity/v3/role_assignments?user.id=8381a7bfacbc41b39631c63b29359220&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:48.838880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6c29-8a83-4adb-981b-737931bd1eac None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:48.839160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6c29-8a83-4adb-981b-737931bd1eac None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:48.839430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6c29-8a83-4adb-981b-737931bd1eac None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:48.841127 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 444/874] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:48.846555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70ecc-818f-4771-ad80-436e42e9dff8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:48.847090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70ecc-818f-4771-ad80-436e42e9dff8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:48.847169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70ecc-818f-4771-ad80-436e42e9dff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:48.868656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c1e70ecc-818f-4771-ad80-436e42e9dff8 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:48.870156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c1e70ecc-818f-4771-ad80-436e42e9dff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:48.940551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 431/875] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:48 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:49.207531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c6c29-8a83-4adb-981b-737931bd1eac None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:49.317481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c6c29-8a83-4adb-981b-737931bd1eac None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:49.328620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:49.352947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d330c9-8b6b-4efa-b50a-2ff46a2d2366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:49.353590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d330c9-8b6b-4efa-b50a-2ff46a2d2366 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:49.353865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d330c9-8b6b-4efa-b50a-2ff46a2d2366 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:49.354135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d330c9-8b6b-4efa-b50a-2ff46a2d2366 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:49.354743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d330c9-8b6b-4efa-b50a-2ff46a2d2366 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:49.356209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d330c9-8b6b-4efa-b50a-2ff46a2d2366 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d1d330c9-8b6b-4efa-b50a-2ff46a2d2366 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:49.365390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:49.366364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:49.367382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 445/876] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:49.376233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70ecc-818f-4771-ad80-436e42e9dff8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:49.419123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:49.473794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c2ee5-ec0a-4a2f-872e-89900ecc48b6 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:49.474450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c2ee5-ec0a-4a2f-872e-89900ecc48b6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:49.474929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c2ee5-ec0a-4a2f-872e-89900ecc48b6 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:49.477790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 446/877] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:22:49.479585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70ecc-818f-4771-ad80-436e42e9dff8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:49.482626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cd216-2f94-4f57-b4e0-1ea3f7d5ba33 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:49.483126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cd216-2f94-4f57-b4e0-1ea3f7d5ba33 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:49.483568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cd216-2f94-4f57-b4e0-1ea3f7d5ba33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:49.502583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-672cd216-2f94-4f57-b4e0-1ea3f7d5ba33 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:49.504208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-672cd216-2f94-4f57-b4e0-1ea3f7d5ba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:49.514245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:49.514965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:49.515304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:49.515637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:49.516337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:49.516697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:49.518134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:49.523375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 432/878] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:22:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:49.530349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:49.586861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 447/879] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:49 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:49.612046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:49.642311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a224f-b413-4491-9cb4-66c4136d251d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:49.678600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586c79b-b224-41ff-9630-f12be744f547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:49.679240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586c79b-b224-41ff-9630-f12be744f547 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:49.679518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586c79b-b224-41ff-9630-f12be744f547 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:49.679786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586c79b-b224-41ff-9630-f12be744f547 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:49.680420 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586c79b-b224-41ff-9630-f12be744f547 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:49.680716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586c79b-b224-41ff-9630-f12be744f547 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:49.682085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586c79b-b224-41ff-9630-f12be744f547 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:49.687070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 433/880] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:22:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:49.857962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cd216-2f94-4f57-b4e0-1ea3f7d5ba33 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:49.960313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cd216-2f94-4f57-b4e0-1ea3f7d5ba33 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:49.963125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:49.997050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025269ce-dcbf-45a8-8111-b00aa306c4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:49.997674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025269ce-dcbf-45a8-8111-b00aa306c4aa None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:49.997949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025269ce-dcbf-45a8-8111-b00aa306c4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:49.998217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025269ce-dcbf-45a8-8111-b00aa306c4aa None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:49.998900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025269ce-dcbf-45a8-8111-b00aa306c4aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:50.000277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025269ce-dcbf-45a8-8111-b00aa306c4aa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:50.025835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 448/881] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:22:49 2022] POST /identity/v3/services => generated 269 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:50.488704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab7f81-2289-4d14-88bd-0b392a4318f3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:50.489265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab7f81-2289-4d14-88bd-0b392a4318f3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:50.489265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab7f81-2289-4d14-88bd-0b392a4318f3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:50.490826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 434/882] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:50.496325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13338c2-e3ab-49fe-9e31-10942dfa6984 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:50.496934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13338c2-e3ab-49fe-9e31-10942dfa6984 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:50.496934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13338c2-e3ab-49fe-9e31-10942dfa6984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:50.515468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a13338c2-e3ab-49fe-9e31-10942dfa6984 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:50.516641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a13338c2-e3ab-49fe-9e31-10942dfa6984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:50.592271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 449/883] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:50.620337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ea80-1d36-4f8b-8609-7ece8e854f44 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:50.620612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ea80-1d36-4f8b-8609-7ece8e854f44 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:50.620895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ea80-1d36-4f8b-8609-7ece8e854f44 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:50.622665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 435/884] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:50.627788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4b62-20c0-4b41-9fb5-f7f8d994c546 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:50.628072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4b62-20c0-4b41-9fb5-f7f8d994c546 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:50.628343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4b62-20c0-4b41-9fb5-f7f8d994c546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:50.646388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-21ad4b62-20c0-4b41-9fb5-f7f8d994c546 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:50.647578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-21ad4b62-20c0-4b41-9fb5-f7f8d994c546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:50.720959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 450/885] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:50.861378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77ea80-1d36-4f8b-8609-7ece8e854f44 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:50.962793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77ea80-1d36-4f8b-8609-7ece8e854f44 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:50.998886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad4b62-20c0-4b41-9fb5-f7f8d994c546 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.000005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.000645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.000915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.001197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.001767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.003172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.005914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:51.012814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:51.013805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:51.014641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 436/886] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:51.021214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.043123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:51.114730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad4b62-20c0-4b41-9fb5-f7f8d994c546 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.135803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b43ed0-499f-4460-8dca-919dc917e770 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.159223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a342f4e-561f-42b5-84b5-235009bcca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.159883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a342f4e-561f-42b5-84b5-235009bcca7c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.160158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a342f4e-561f-42b5-84b5-235009bcca7c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.160424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a342f4e-561f-42b5-84b5-235009bcca7c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.174053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a342f4e-561f-42b5-84b5-235009bcca7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:51.179437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4a342f4e-561f-42b5-84b5-235009bcca7c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:51.179589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:51.180469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:51.181282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 451/887] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 31 04:22:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:51.186969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.186969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.186969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.186969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:51.186969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.186969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.187491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a342f4e-561f-42b5-84b5-235009bcca7c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.193525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 437/888] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:51.198696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.273076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:51.280384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:51.301173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a342f4e-561f-42b5-84b5-235009bcca7c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.322950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605414-c881-42b1-87c8-a55bcb81161f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.341842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.342885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.343358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.343816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.344692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:51.345181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.346813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.355762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 452/889] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:51.361469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.369523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.370073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.370346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.370613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.371408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:51.371704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.373129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.383334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 438/890] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:51.384202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.490427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a876df1-7d6d-4307-81a1-ac05fb7db38d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.496976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ae142-a8f2-4753-9cd6-de0f6b724d39 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.519287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:51.535498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.536110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.536383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.536649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.537358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] RBAC: Adding query filter params (interface=public, service_id=e4af3ebb07e24207ad59a42a075ae52e, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:51.537659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.539049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.547760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.548702 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.549275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.549603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 439/891] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/endpoints?service_id=e4af3ebb07e24207ad59a42a075ae52e&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:51.549963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.550480 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.563631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.564970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.567021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:51.583085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:51.584082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:51.584944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:51.584944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:51.584944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:51.589475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 453/892] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:51.590455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.667893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb42882-d0ce-4a99-89d9-f6524c37bd6f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.704159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac676b-3953-493e-b5aa-746a687fc130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.704760 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac676b-3953-493e-b5aa-746a687fc130 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.705036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac676b-3953-493e-b5aa-746a687fc130 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.705302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac676b-3953-493e-b5aa-746a687fc130 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.705901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac676b-3953-493e-b5aa-746a687fc130 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.706697 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672685a-8e87-4d4b-ad87-81c66de51143 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.707296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac676b-3953-493e-b5aa-746a687fc130 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.718769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 440/893] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:22:51.718769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf14b7-fcd4-4b84-a815-17a5e82d5848 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.718769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf14b7-fcd4-4b84-a815-17a5e82d5848 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.718769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf14b7-fcd4-4b84-a815-17a5e82d5848 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.718769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 441/894] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:51.721359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b928-f032-48d7-b86c-a60ddf78b2e3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.721676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b928-f032-48d7-b86c-a60ddf78b2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.722330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b928-f032-48d7-b86c-a60ddf78b2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.743919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c084b928-f032-48d7-b86c-a60ddf78b2e3 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:51.745516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c084b928-f032-48d7-b86c-a60ddf78b2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:51.751962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.752680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.753023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.753358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.754106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:51.754466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.756112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.775552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 454/895] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:51.776562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.802425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:51.831274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 442/896] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:22:51.839758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:51.890459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73008726-6d47-4d4d-8c9a-18b5d3425ff0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:51.924971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07fc53-6820-4958-ad69-71a8ebb4cb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:51.925537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07fc53-6820-4958-ad69-71a8ebb4cb4e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.925810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07fc53-6820-4958-ad69-71a8ebb4cb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.926071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07fc53-6820-4958-ad69-71a8ebb4cb4e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.933519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07fc53-6820-4958-ad69-71a8ebb4cb4e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:51.935008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07fc53-6820-4958-ad69-71a8ebb4cb4e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-df07fc53-6820-4958-ad69-71a8ebb4cb4e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:51.947015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:51.947946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:51.948874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 455/897] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:51.955835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084b928-f032-48d7-b86c-a60ddf78b2e3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:51.985097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524fafe-8326-4ae4-a53e-cff3726d899d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.985413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524fafe-8326-4ae4-a53e-cff3726d899d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.985632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524fafe-8326-4ae4-a53e-cff3726d899d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:51.987313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 456/898] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:22:51.993873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ad79-2434-4f56-ac2d-d9b4dd0302a6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:51.994153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ad79-2434-4f56-ac2d-d9b4dd0302a6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:51.994426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ad79-2434-4f56-ac2d-d9b4dd0302a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.013021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-be14ad79-2434-4f56-ac2d-d9b4dd0302a6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:52.014221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-be14ad79-2434-4f56-ac2d-d9b4dd0302a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:52.078946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084b928-f032-48d7-b86c-a60ddf78b2e3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:52.104089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 457/899] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:22:52.105797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14ad79-2434-4f56-ac2d-d9b4dd0302a6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.116522 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:52.117409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:52.117871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:52.118306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.119257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:52.119737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:52.121306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:52.139389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 443/900] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 31 04:22:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:52.144680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.161785 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:52.195249 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:52.252499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14ad79-2434-4f56-ac2d-d9b4dd0302a6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:52.261879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccca715-b2e1-4474-a436-6cd04d392efe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:52.297151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c04683-f7e4-4364-a524-d8ab699755a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:52.297863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c04683-f7e4-4364-a524-d8ab699755a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:52.298133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c04683-f7e4-4364-a524-d8ab699755a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:52.298402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c04683-f7e4-4364-a524-d8ab699755a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.299198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53fcea-7148-430f-91ac-824bd04c3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:52.300065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fcea-7148-430f-91ac-824bd04c3663 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:52.300539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fcea-7148-430f-91ac-824bd04c3663 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:52.301270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fcea-7148-430f-91ac-824bd04c3663 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.303075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53fcea-7148-430f-91ac-824bd04c3663 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:52.304845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53fcea-7148-430f-91ac-824bd04c3663 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:52.327335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 458/901] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:22:52 2022] POST /identity/v3/services => generated 241 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:52.387711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53fcea-7148-430f-91ac-824bd04c3663 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.393070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c04683-f7e4-4364-a524-d8ab699755a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:52.394961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c04683-f7e4-4364-a524-d8ab699755a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:52.462293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:52.506780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:52.511471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53fcea-7148-430f-91ac-824bd04c3663 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:52.540565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 444/902] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:22:52 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:22:52.546569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c04683-f7e4-4364-a524-d8ab699755a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.555109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:52.555755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:52.556031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:52.556300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.556892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:52.558278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:52.567821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:52.569124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:52.569995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:52.573738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 459/903] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Aug 31 04:22:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:52.575501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.668798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c04683-f7e4-4364-a524-d8ab699755a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:52.696140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8153dc-cb00-4520-b20b-19ffd911da06 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:52.704478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:52.705113 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:52.705389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:52.705653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.716532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:52.717976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:52.727299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:52.736483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:52.739023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:52.740387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:52.740387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:52.740387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:52.740387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 445/904] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:52.742303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:52.743996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:52.743996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:52.743996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.744261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.744337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:52.744538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:52.745911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:52.758803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 460/905] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:52.762468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.801770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:52.859416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a47f5-f07e-4af2-9b79-043377a7d331 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:52.896352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848db8a-30ff-4054-85c7-ec2d1cc19505 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:52.901841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:52.902502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:52.902807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:52.903075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.903729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:52.904031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:52.905407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:52.918476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 446/906] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:52.923910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.934950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:52.935938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:52.936405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:52.936923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:52.937755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:52.938155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:52.939753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:52.945802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 461/907] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:52.951869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:52.987678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:53.042789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503e099-52c6-49d9-b0b2-af02136514ff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:53.051037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:53.066392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7d8f9-33fb-4f61-81ce-f258ae11da12 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:53.100278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:53.101013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:53.101297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:53.101567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:53.103437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:53.104274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:53.104776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:53.105230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:53.106245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] RBAC: Adding query filter params (interface=public, service_id=dfdc8075c2b24aa0843ca95428a94958, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:53.106779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:53.108338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:53.109201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:53.110915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:53.118927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 462/908] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 31 04:22:52 2022] GET /identity/v3/endpoints?service_id=dfdc8075c2b24aa0843ca95428a94958&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:53.120066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:53.123920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:53.124870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:53.125737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:53.125737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:53.125737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:53.125737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:53.125737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:53.125737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:53.125737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:53.125737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:53.130799 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 447/909] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:53.132141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:53.233413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64311a-c68f-4f6a-bf05-d0efb815c430 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:53.269361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bbb0-6cb4-413d-80ca-00453cc4e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:53.269887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bbb0-6cb4-413d-80ca-00453cc4e8ac None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:53.270160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bbb0-6cb4-413d-80ca-00453cc4e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:53.270428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bbb0-6cb4-413d-80ca-00453cc4e8ac None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:53.271126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d47ca-a77e-4f98-864d-2ffd72d431a6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:53.271260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bbb0-6cb4-413d-80ca-00453cc4e8ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:53.272519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bbb0-6cb4-413d-80ca-00453cc4e8ac None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:53.283124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 463/910] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:22:53 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:22:53.290916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:53.312852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b08d6-ae67-4a8d-a159-9c5852cfd6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:53.313905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b08d6-ae67-4a8d-a159-9c5852cfd6ee None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:53.314394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b08d6-ae67-4a8d-a159-9c5852cfd6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:53.314899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b08d6-ae67-4a8d-a159-9c5852cfd6ee None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:53.315841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b08d6-ae67-4a8d-a159-9c5852cfd6ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:53.316348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b08d6-ae67-4a8d-a159-9c5852cfd6ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:53.318058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b08d6-ae67-4a8d-a159-9c5852cfd6ee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:53.338365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 448/911] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:53.340416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bbb0-6cb4-413d-80ca-00453cc4e8ac None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:53.365980 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:53.468185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bbb0-6cb4-413d-80ca-00453cc4e8ac None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:53.519977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5ecb-50ce-46d3-8210-9cce592b1a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:53.520619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5ecb-50ce-46d3-8210-9cce592b1a05 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:53.520849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5ecb-50ce-46d3-8210-9cce592b1a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:53.521045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5ecb-50ce-46d3-8210-9cce592b1a05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:53.521786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5ecb-50ce-46d3-8210-9cce592b1a05 None admin] RBAC: Adding query filter params (role.id=d15c78e6c27741619010a21e9a172da7, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=8381a7bfacbc41b39631c63b29359220) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:53.522091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5ecb-50ce-46d3-8210-9cce592b1a05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:53.523626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5ecb-50ce-46d3-8210-9cce592b1a05 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:53.530356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 464/912] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:22:53 2022] GET /identity/v3/role_assignments?user.id=8381a7bfacbc41b39631c63b29359220&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=d15c78e6c27741619010a21e9a172da7 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:53.717924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e114c87-0de0-414f-8441-d171c2919852 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:53.718453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e114c87-0de0-414f-8441-d171c2919852 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:53.718453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e114c87-0de0-414f-8441-d171c2919852 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:53.720075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 449/913] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:53.725218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2939d-a830-4bb8-a1de-ab268006441d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:53.725502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2939d-a830-4bb8-a1de-ab268006441d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:53.725773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2939d-a830-4bb8-a1de-ab268006441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:53.744290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-94b2939d-a830-4bb8-a1de-ab268006441d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:53.745468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-94b2939d-a830-4bb8-a1de-ab268006441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:53.814143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 465/914] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:54.077569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e114c87-0de0-414f-8441-d171c2919852 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:54.183064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e114c87-0de0-414f-8441-d171c2919852 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:54.219582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d8719-ee9e-4e9a-8111-b3b27d3b2882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:54.220583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d8719-ee9e-4e9a-8111-b3b27d3b2882 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:54.221085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d8719-ee9e-4e9a-8111-b3b27d3b2882 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:54.221573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d8719-ee9e-4e9a-8111-b3b27d3b2882 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:54.222378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d8719-ee9e-4e9a-8111-b3b27d3b2882 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:54.223987 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d8719-ee9e-4e9a-8111-b3b27d3b2882 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-407d8719-ee9e-4e9a-8111-b3b27d3b2882 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:54.233767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:54.234798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 04:22:54.235614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:54.238465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 450/915] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:54.244415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2939d-a830-4bb8-a1de-ab268006441d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:54.259986 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:54.347851 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2939d-a830-4bb8-a1de-ab268006441d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:54.381301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1030c-7f7f-435f-8659-b700ec10214b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:54.381584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1030c-7f7f-435f-8659-b700ec10214b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:54.381863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1030c-7f7f-435f-8659-b700ec10214b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:54.383373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 451/916] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:22:54.385413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:54.386024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:54.386298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:54.386571 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:54.387604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:54.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:54.389277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:54.391342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31b5a2-d003-46d1-bc52-5cf455e0dd52 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:54.391888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31b5a2-d003-46d1-bc52-5cf455e0dd52 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:54.392231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31b5a2-d003-46d1-bc52-5cf455e0dd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:54.398449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 466/917] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:54.407834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:54.412639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2a31b5a2-d003-46d1-bc52-5cf455e0dd52 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:54.413814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a31b5a2-d003-46d1-bc52-5cf455e0dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:54.452347 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:54.489587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 452/918] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:54.500503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:54.516679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae79e2-2888-49a0-a0ac-7ca3874b6db0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:54.551547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:54.552204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:54.552430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:54.552695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:54.553337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:54.553634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:54.555040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:54.560938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 467/919] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Aug 31 04:22:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:54.566406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31b5a2-d003-46d1-bc52-5cf455e0dd52 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:54.674963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31b5a2-d003-46d1-bc52-5cf455e0dd52 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:54.722622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92593-4140-444f-bcfd-238249e9e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:54.723336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92593-4140-444f-bcfd-238249e9e1bd None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:54.723677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92593-4140-444f-bcfd-238249e9e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:54.723777 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92593-4140-444f-bcfd-238249e9e1bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:54.725031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d92593-4140-444f-bcfd-238249e9e1bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:54.725806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d92593-4140-444f-bcfd-238249e9e1bd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:54.766094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:54.773690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 453/920] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:22:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:54.920967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e84b95-7cbe-4281-a773-c14bd8cdb3b6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:54.969320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:54.969968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:54.970250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:54.970523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:54.971138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:54.972510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:54.983249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:54.984245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:54.985068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 468/921] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:54.991832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92593-4140-444f-bcfd-238249e9e1bd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:55.030149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:55.111379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92593-4140-444f-bcfd-238249e9e1bd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:55.155488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:55.156119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.156395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.156657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.157298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:55.157590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:55.158975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:55.172243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 454/922] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:22:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:55.183148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:55.193175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:55.307535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e5e4a-7961-42d3-a43f-4c6f7cdccb32 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:55.349630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25d1e3-392a-4910-a47d-819b466aed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:55.350252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25d1e3-392a-4910-a47d-819b466aed48 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.350531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25d1e3-392a-4910-a47d-819b466aed48 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.350834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25d1e3-392a-4910-a47d-819b466aed48 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.351468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25d1e3-392a-4910-a47d-819b466aed48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:55.351765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25d1e3-392a-4910-a47d-819b466aed48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:55.353144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25d1e3-392a-4910-a47d-819b466aed48 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:55.358738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 469/923] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:22:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:55.364268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:55.374132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf2d94-a831-4c4b-8b43-e1ba77b49da7 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.374434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf2d94-a831-4c4b-8b43-e1ba77b49da7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.374691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf2d94-a831-4c4b-8b43-e1ba77b49da7 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.376332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 470/924] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:55.381244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940420-d242-459d-8eb5-9723bfcfff27 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.381523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940420-d242-459d-8eb5-9723bfcfff27 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.381801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940420-d242-459d-8eb5-9723bfcfff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.400114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-49940420-d242-459d-8eb5-9723bfcfff27 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:55.401308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-49940420-d242-459d-8eb5-9723bfcfff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:55.474348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:55.480932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:55.491698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97528c77-df7e-40bd-9853-6894c9a48041 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:55.498570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 471/925] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:22:55.535211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c55f8-b738-436f-a5fe-8873cba6e942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:55.535823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c55f8-b738-436f-a5fe-8873cba6e942 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.536099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c55f8-b738-436f-a5fe-8873cba6e942 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.536370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c55f8-b738-436f-a5fe-8873cba6e942 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.537086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c55f8-b738-436f-a5fe-8873cba6e942 None admin] RBAC: Adding query filter params (interface=public, service_id=bf0aa09d380543ca9572fec3ed6f0ce3, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:55.537440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c55f8-b738-436f-a5fe-8873cba6e942 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:55.538840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c55f8-b738-436f-a5fe-8873cba6e942 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:55.544669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 455/926] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 31 04:22:55 2022] GET /identity/v3/endpoints?service_id=bf0aa09d380543ca9572fec3ed6f0ce3&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:55.549961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49940420-d242-459d-8eb5-9723bfcfff27 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:55.650963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49940420-d242-459d-8eb5-9723bfcfff27 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:55.680186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd72ee-c0bd-4d2c-b4c1-0cd6385006ff None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.680511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd72ee-c0bd-4d2c-b4c1-0cd6385006ff None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.680738 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd72ee-c0bd-4d2c-b4c1-0cd6385006ff None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.682311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 456/927] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:55.685604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee145cb-7863-4813-bd96-30ba0efeb1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:55.686296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee145cb-7863-4813-bd96-30ba0efeb1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.686578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee145cb-7863-4813-bd96-30ba0efeb1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.686882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee145cb-7863-4813-bd96-30ba0efeb1f8 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.687316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b349d2-72d1-44cf-9f99-7acf6854fbf5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.687483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee145cb-7863-4813-bd96-30ba0efeb1f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:55.687579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b349d2-72d1-44cf-9f99-7acf6854fbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.687848 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b349d2-72d1-44cf-9f99-7acf6854fbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.688858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee145cb-7863-4813-bd96-30ba0efeb1f8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:55.701590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 472/928] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:22:55 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:22:55.706762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-09b349d2-72d1-44cf-9f99-7acf6854fbf5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:55.707989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-09b349d2-72d1-44cf-9f99-7acf6854fbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:55.780140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 457/929] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:22:55.787179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee145cb-7863-4813-bd96-30ba0efeb1f8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:55.855019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:55.903086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee145cb-7863-4813-bd96-30ba0efeb1f8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:55.940119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:55.940762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:55.941038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:55.941305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:55.941921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:55.943704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:55.952735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:55.953744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:55.954738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 473/930] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Aug 31 04:22:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:55.960806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b349d2-72d1-44cf-9f99-7acf6854fbf5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:56.055439 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:56.064245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b349d2-72d1-44cf-9f99-7acf6854fbf5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:56.073825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:56.105192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:56.105944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.106221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.106500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.107176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:56.107476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:56.108868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:56.118675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 458/931] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:56.120313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:56.199897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9c212-4d26-4b19-bffd-2030e26b2d73 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:56.229980 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:56.243180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08a2b-a8e4-41e3-9c2f-76332625a800 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:56.244145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e69bed-ccdf-4c47-b890-ea2fb49a9031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:56.244784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69bed-ccdf-4c47-b890-ea2fb49a9031 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.245058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69bed-ccdf-4c47-b890-ea2fb49a9031 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.245324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69bed-ccdf-4c47-b890-ea2fb49a9031 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.258421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e69bed-ccdf-4c47-b890-ea2fb49a9031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-36e69bed-ccdf-4c47-b890-ea2fb49a9031 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:22:56.265227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:56.266180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:56.267224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 474/932] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 31 04:22:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:56.272186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e69bed-ccdf-4c47-b890-ea2fb49a9031 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:56.294697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:56.296125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.296851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.297304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.298375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:56.299278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:56.301419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:56.313783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 459/933] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:56.316426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:56.335941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:56.388085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e69bed-ccdf-4c47-b890-ea2fb49a9031 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:56.428736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:56.429902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.430334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.430799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.431872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:56.432691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:56.434356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:56.447273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 475/934] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 31 04:22:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:56.453052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:56.454283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74350670-de2c-4d0d-97a3-e78e202d721a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:56.490366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d9610-792f-4847-b5d7-78bede164e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:56.491094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d9610-792f-4847-b5d7-78bede164e63 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.491436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d9610-792f-4847-b5d7-78bede164e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.491775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d9610-792f-4847-b5d7-78bede164e63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.492562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d9610-792f-4847-b5d7-78bede164e63 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:56.494004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d9610-792f-4847-b5d7-78bede164e63 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:56.516390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:56.523006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 460/935] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:22:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:22:56.582684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a10c4-2dc8-4f1e-90ec-89825a67104f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:56.627436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:56.628396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.628871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.629321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.643034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:56.644849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:56.660973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:56.662450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:56.663391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:56.663391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:56.663391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:56.665030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 476/936] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:56.671340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d9610-792f-4847-b5d7-78bede164e63 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:56.743358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:56.779587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d9610-792f-4847-b5d7-78bede164e63 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:56.814442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e017362-2246-4b79-a463-1dbc35999efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:56.815129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e017362-2246-4b79-a463-1dbc35999efb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.815370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e017362-2246-4b79-a463-1dbc35999efb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.815602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e017362-2246-4b79-a463-1dbc35999efb None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.816260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e017362-2246-4b79-a463-1dbc35999efb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:56.816556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e017362-2246-4b79-a463-1dbc35999efb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:56.817923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e017362-2246-4b79-a463-1dbc35999efb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:56.830907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 461/937] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:56.836721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:56.935891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7d62-d1d0-454e-b10c-302699f281fa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.936238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7d62-d1d0-454e-b10c-302699f281fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.936438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7d62-d1d0-454e-b10c-302699f281fa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.937767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 462/938] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:22:56.943954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:56.944386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc052-a18d-473c-81b8-73023553cd9b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.944853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc052-a18d-473c-81b8-73023553cd9b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.945202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc052-a18d-473c-81b8-73023553cd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:56.951895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975118c0-3557-4954-964e-e39d11f0b09c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:56.972489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2a5bc052-a18d-473c-81b8-73023553cd9b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:56.974370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a5bc052-a18d-473c-81b8-73023553cd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:56.993119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:56.994112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:56.994575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:56.995054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.003410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.005146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:22:57.017033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:57.018032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:22:57.019016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.021605 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 477/939] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:22:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:57.027408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:57.054782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 463/940] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:22:57.141699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2188f-82eb-4909-8048-a4ee14fc470e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:57.176684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:57.177564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.177987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.178393 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.179292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:57.179749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.181312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.200267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 478/941] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 31 04:22:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:57.206781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc052-a18d-473c-81b8-73023553cd9b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:57.264761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:57.310403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc052-a18d-473c-81b8-73023553cd9b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:57.340305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:57.348878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:57.349519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.349794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.350061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.358770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:57.365781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.367333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:22:57.387909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:57.388927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:22:57.395018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:22:57.395018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:22:57.395018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:57.395018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 464/942] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:57.396590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:57.469876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2135b4-f6cc-4b37-a520-b7cd68d13038 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:57.505474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:57.505950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.506231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.506500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.507194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.508654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.511282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING keystone.server.flask.common [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 04:22:57.512890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b645103-27df-4ec7-a816-9140a09dea7a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:57.527038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:57.530132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.sql.core [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 04:22:57.530132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 04:22:57.530132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [parameters: {'id': '31300252160e487eb4edbd8d03c9c8d0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 04:22:57.530132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81569) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 31300252160e487eb4edbd8d03c9c8d0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 31300252160e487eb4edbd8d03c9c8d0. Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application conn.query(q) Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 04:22:57.549680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result.read() Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 04:22:57.550582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application next(self.gen) Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application yield resource Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application next(self.gen) Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise self.value Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application session.commit() Aug 31 04:22:57.551557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.session.flush() Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application compat.raise_( Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 04:22:57.552462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 04:22:57.553328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application conn.query(q) Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result.read() Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 04:22:57.554089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [parameters: {'id': '31300252160e487eb4edbd8d03c9c8d0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:57.554953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:57.555868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 31300252160e487eb4edbd8d03c9c8d0. Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.557126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 479/943] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Aug 31 04:22:57 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:57.559522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:57.561617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:57.562416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.562980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.563399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.564290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:57.564718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.566236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.580299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 465/944] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:22:57.587693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:57.634849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:57.673480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0b810-b3ae-4e8e-b3e9-c2a4640ca5a8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:57.702140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de2d03-ee67-4db9-b311-9f6892dbef57 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:57.712144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:57.713008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.713383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.713771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.721321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.723231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:57.737913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:57.739100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:57.740031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:57.740031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:57.740031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:57.740031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:57.740031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:57.740031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:57.740031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:57.740031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:57.742796 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 480/945] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:57.746373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd84aee-93c1-4c3e-a801-d486d30433fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:57.747044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd84aee-93c1-4c3e-a801-d486d30433fc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.747319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd84aee-93c1-4c3e-a801-d486d30433fc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.748364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:57.748877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd84aee-93c1-4c3e-a801-d486d30433fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.756266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd84aee-93c1-4c3e-a801-d486d30433fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.757709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd84aee-93c1-4c3e-a801-d486d30433fc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3bd84aee-93c1-4c3e-a801-d486d30433fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:22:57.770840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:57.772012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:22:57.773352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 466/946] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:57.776760 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834e935-d07e-4a38-8d61-562f70727299 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.777102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834e935-d07e-4a38-8d61-562f70727299 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.777973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834e935-d07e-4a38-8d61-562f70727299 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.779528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 467/947] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:57.780223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834e935-d07e-4a38-8d61-562f70727299 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:57.812002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:57.861120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877653d-1c6c-43cc-8909-29c2001a536e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:57.890167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834e935-d07e-4a38-8d61-562f70727299 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:57.892060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:57.902161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0919d0-ac20-451c-bc0e-ffb0b3b609c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:57.903048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0919d0-ac20-451c-bc0e-ffb0b3b609c2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.903327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0919d0-ac20-451c-bc0e-ffb0b3b609c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.903597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0919d0-ac20-451c-bc0e-ffb0b3b609c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.904265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0919d0-ac20-451c-bc0e-ffb0b3b609c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:57.904561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0919d0-ac20-451c-bc0e-ffb0b3b609c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.905995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0919d0-ac20-451c-bc0e-ffb0b3b609c2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.925570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:57.926185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.926459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.935088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.935331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 481/948] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:57.935424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:57.935424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:57.935552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da42d-5b1f-4051-9964-25ea3e766895 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:57.935552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da42d-5b1f-4051-9964-25ea3e766895 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:57.935552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da42d-5b1f-4051-9964-25ea3e766895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:57.935754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:57.949483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d19da42d-5b1f-4051-9964-25ea3e766895 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:57.950858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 468/949] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:22:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:57.951192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d19da42d-5b1f-4051-9964-25ea3e766895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:57.956894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:58.046979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 482/950] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:22:58.086397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:58.092959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b2818-f185-49d2-ba66-80bc955eb1db None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:58.129158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fa48-108f-411e-9bd3-bb1601a37ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:58.129819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fa48-108f-411e-9bd3-bb1601a37ff6 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:58.130092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fa48-108f-411e-9bd3-bb1601a37ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:58.130358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fa48-108f-411e-9bd3-bb1601a37ff6 None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/8381a7bfacbc41b39631c63b29359220/roles/d15c78e6c27741619010a21e9a172da7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:58.132529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4fa48-108f-411e-9bd3-bb1601a37ff6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=8381a7bfacbc41b39631c63b29359220, role_id=d15c78e6c27741619010a21e9a172da7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:58.133962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4fa48-108f-411e-9bd3-bb1601a37ff6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:58.152376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 469/951] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:22:57 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/8381a7bfacbc41b39631c63b29359220/roles/d15c78e6c27741619010a21e9a172da7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:22:58.320501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19da42d-5b1f-4051-9964-25ea3e766895 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:58.446536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19da42d-5b1f-4051-9964-25ea3e766895 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:58.465154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:58.483187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1569f375-c80a-4270-b99d-d787067f88a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:58.483832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569f375-c80a-4270-b99d-d787067f88a9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:58.484111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569f375-c80a-4270-b99d-d787067f88a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:58.484376 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569f375-c80a-4270-b99d-d787067f88a9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:58.484964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1569f375-c80a-4270-b99d-d787067f88a9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:58.486424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1569f375-c80a-4270-b99d-d787067f88a9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-1569f375-c80a-4270-b99d-d787067f88a9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:58.495603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:58.496721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:58.497973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 483/952] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:22:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:22:58.503962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a18d7-d46d-4a22-a6dc-6167e5924217 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:58.504251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a18d7-d46d-4a22-a6dc-6167e5924217 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:58.504528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a18d7-d46d-4a22-a6dc-6167e5924217 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:58.505083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fa48-108f-411e-9bd3-bb1601a37ff6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:58.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 484/953] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:22:58.511967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b6b89-0c4c-40ac-9e38-e1511a9ebf8e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:58.512259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b6b89-0c4c-40ac-9e38-e1511a9ebf8e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:58.512530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b6b89-0c4c-40ac-9e38-e1511a9ebf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:58.531016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-497b6b89-0c4c-40ac-9e38-e1511a9ebf8e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:22:58.532275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-497b6b89-0c4c-40ac-9e38-e1511a9ebf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:58.611080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 485/954] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:22:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:22:58.627769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fa48-108f-411e-9bd3-bb1601a37ff6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:58.661586 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:58.662575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:58.663090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:58.663567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:58.664390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:58.664887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:58.666463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:58.671974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 470/955] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:22:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:58.680314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b6b89-0c4c-40ac-9e38-e1511a9ebf8e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:58.795624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b6b89-0c4c-40ac-9e38-e1511a9ebf8e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:58.809245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:58.831167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:58.832106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:58.832600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:58.833078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:58.833897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:58.834387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:58.835986 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:58.841634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 486/956] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:22:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:58.847821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:58.865290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:58.915137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:58.956631 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ada183-f182-47af-8564-69a89ee2100c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:58.993758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71e9a3-c0a6-4b49-bf18-336766426b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:58.994401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71e9a3-c0a6-4b49-bf18-336766426b44 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:58.994680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71e9a3-c0a6-4b49-bf18-336766426b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:58.994993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71e9a3-c0a6-4b49-bf18-336766426b44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:58.995832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71e9a3-c0a6-4b49-bf18-336766426b44 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:58.997450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71e9a3-c0a6-4b49-bf18-336766426b44 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:59.020771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda56e04-d038-4e03-af76-cca3484e5050 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:59.022798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 471/957] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:22:58 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:22:59.078628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:59.079331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:59.079695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:59.079970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] PATH_INFO: `/v3/services/compute` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:59.080573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:59.082113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:22:59.092258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:22:59.093351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:22:59.094627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 487/958] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:22:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:22:59.099904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71e9a3-c0a6-4b49-bf18-336766426b44 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:59.179278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:59.212652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71e9a3-c0a6-4b49-bf18-336766426b44 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:59.247665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da1370-641c-4875-9faf-48b9947c4e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:59.248281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da1370-641c-4875-9faf-48b9947c4e82 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:59.248555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da1370-641c-4875-9faf-48b9947c4e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:59.248819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da1370-641c-4875-9faf-48b9947c4e82 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:59.249451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da1370-641c-4875-9faf-48b9947c4e82 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:59.249746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da1370-641c-4875-9faf-48b9947c4e82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:59.251165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da1370-641c-4875-9faf-48b9947c4e82 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:59.256236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 472/959] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:22:59.262009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:59.304847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:59.375769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ee98-5cb8-4a70-a9dd-2d0296d220ce None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:59.409815 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb20235-544d-44cc-9ae1-af3e3d2f12af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:59.410424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb20235-544d-44cc-9ae1-af3e3d2f12af None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:59.410698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb20235-544d-44cc-9ae1-af3e3d2f12af None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:59.410989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb20235-544d-44cc-9ae1-af3e3d2f12af None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:59.411621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb20235-544d-44cc-9ae1-af3e3d2f12af None admin] RBAC: Adding query filter params (type=compute) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:22:59.411915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb20235-544d-44cc-9ae1-af3e3d2f12af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:59.413279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb20235-544d-44cc-9ae1-af3e3d2f12af None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:59.418206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 488/960] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:22:59 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:00.018698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811ccb-c30f-4452-b273-e4c21303573a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.019738 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811ccb-c30f-4452-b273-e4c21303573a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.020236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4811ccb-c30f-4452-b273-e4c21303573a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.022073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 473/961] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:00.027638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06074be3-f754-480d-b201-9dbebe2691d5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.028439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06074be3-f754-480d-b201-9dbebe2691d5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.028926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06074be3-f754-480d-b201-9dbebe2691d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.048204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-06074be3-f754-480d-b201-9dbebe2691d5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:00.049760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-06074be3-f754-480d-b201-9dbebe2691d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:00.130823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 489/962] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:00.161965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1386-3d4e-4335-9e91-57b2ca9099df None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.162560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1386-3d4e-4335-9e91-57b2ca9099df None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.163078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1386-3d4e-4335-9e91-57b2ca9099df None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.164881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 474/963] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:00.172346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4ada6-d9ee-4b4a-837d-74c0960a3c67 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.172885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4ada6-d9ee-4b4a-837d-74c0960a3c67 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.173349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4ada6-d9ee-4b4a-837d-74c0960a3c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.192810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-42a4ada6-d9ee-4b4a-837d-74c0960a3c67 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:00.194786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-42a4ada6-d9ee-4b4a-837d-74c0960a3c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:00.221066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:00.270918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 490/964] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:00.417968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b1386-3d4e-4335-9e91-57b2ca9099df None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:00.530772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b1386-3d4e-4335-9e91-57b2ca9099df None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:00.536843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4ada6-d9ee-4b4a-837d-74c0960a3c67 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:00.566885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:00.567501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.567780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.568053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.569060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:00.570504 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:00.573086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING keystone.server.flask.common [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 04:23:00.580540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:00.588232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.sql.core [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 04:23:00.588232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 04:23:00.588232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [parameters: {'user_id': 'fd3668c1420e48c89e049cc2d947ae1e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 04:23:00.588232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81570) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Aug 31 04:23:00.595910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 04:23:00.596815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application yield resource Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise self.value Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application session.commit() Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 04:23:00.598190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.flush() Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application compat.raise_( Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 04:23:00.599331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 04:23:00.600208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 04:23:00.601048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fd3668c1420e48c89e049cc2d947ae1e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:00.601869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 04:23:00.604374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 04:23:00.606826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 04:23:00.606826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 04:23:00.606826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 04:23:00.606826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 04:23:00.606826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 04:23:00.606826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:00.606826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 475/965] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Aug 31 04:23:00 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:00.606826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:00.664942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4ada6-d9ee-4b4a-837d-74c0960a3c67 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:00.709067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ebcc2-13ea-4e07-ab7c-c5926b4ed189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:00.710440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ebcc2-13ea-4e07-ab7c-c5926b4ed189 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.710967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ebcc2-13ea-4e07-ab7c-c5926b4ed189 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.711750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ebcc2-13ea-4e07-ab7c-c5926b4ed189 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.718572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97250f5-3207-47f1-8605-00a4dd77e054 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:00.726054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ebcc2-13ea-4e07-ab7c-c5926b4ed189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-557ebcc2-13ea-4e07-ab7c-c5926b4ed189 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:00.733491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:00.734470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:23:00.735362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:00.737722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 491/966] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 31 04:23:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:23:00.743175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ebcc2-13ea-4e07-ab7c-c5926b4ed189 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:00.764611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:00.765253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.765534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.765860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.771167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:00.774339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:00.776141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:23:00.788812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:00.789773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 04:23:00.791388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:00.794937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 476/967] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:23:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:23:00.795821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:00.854038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ebcc2-13ea-4e07-ab7c-c5926b4ed189 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:00.854782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:00.892292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:00.893315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.893789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.894255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.895165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:00.895660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:00.897269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:00.904163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b68d76-3f95-433b-9802-6fae9f525916 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:00.911675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 492/968] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 31 04:23:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:00.913244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:00.940579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039edab-4d5e-4b92-8d41-734491cd02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:00.941407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039edab-4d5e-4b92-8d41-734491cd02d5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:00.941518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039edab-4d5e-4b92-8d41-734491cd02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:00.941775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039edab-4d5e-4b92-8d41-734491cd02d5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:00.942498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039edab-4d5e-4b92-8d41-734491cd02d5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:00.942817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039edab-4d5e-4b92-8d41-734491cd02d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:00.944620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039edab-4d5e-4b92-8d41-734491cd02d5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:00.990916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 477/969] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Aug 31 04:23:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:01.004258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:01.026182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59800-00fb-4c43-8807-183e03ffb61c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:01.038324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dba45-2f5f-4f2f-ac88-f22f55fa4713 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.038994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dba45-2f5f-4f2f-ac88-f22f55fa4713 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.038994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dba45-2f5f-4f2f-ac88-f22f55fa4713 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.040768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 478/970] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:23:01.046660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad98e3-a837-4a91-8924-219c9990bb1e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.046997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad98e3-a837-4a91-8924-219c9990bb1e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.047236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad98e3-a837-4a91-8924-219c9990bb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.067527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-94ad98e3-a837-4a91-8924-219c9990bb1e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:01.069514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-94ad98e3-a837-4a91-8924-219c9990bb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:01.074021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:01.075668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.076174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.076638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.088844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:01.090811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:01.107046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:01.108079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:01.108969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:01.108969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:01.108969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:01.115870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 493/971] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:23:01.117591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:01.150884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 479/972] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:23:01.224457 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e8a9b-4596-4870-87d7-dc6f61056d66 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:01.259745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81f2c-65a1-4bee-87b2-de01866e7086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:01.260727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81f2c-65a1-4bee-87b2-de01866e7086 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.261204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81f2c-65a1-4bee-87b2-de01866e7086 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.261672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81f2c-65a1-4bee-87b2-de01866e7086 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.262931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81f2c-65a1-4bee-87b2-de01866e7086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:01.263441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81f2c-65a1-4bee-87b2-de01866e7086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:01.265039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81f2c-65a1-4bee-87b2-de01866e7086 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:01.266937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:01.279526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 494/973] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:01.284132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad98e3-a837-4a91-8924-219c9990bb1e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:01.317623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134e6e-bd99-4978-b240-dbff7adb5fdb None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.317922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134e6e-bd99-4978-b240-dbff7adb5fdb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.318204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134e6e-bd99-4978-b240-dbff7adb5fdb None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.321410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 495/974] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:01.326406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddb1ca-db29-4f02-8962-47b37531484f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.326676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddb1ca-db29-4f02-8962-47b37531484f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.327005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddb1ca-db29-4f02-8962-47b37531484f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.335137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:01.346958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b0ddb1ca-db29-4f02-8962-47b37531484f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:01.348152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b0ddb1ca-db29-4f02-8962-47b37531484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:01.394459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad98e3-a837-4a91-8924-219c9990bb1e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:01.419806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 496/975] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:23:01.420912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddb1ca-db29-4f02-8962-47b37531484f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:01.435823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:01.436463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.436737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.437001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.444803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:01.446236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:23:01.457847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:01.458818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:01.464765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 480/976] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:23:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:01.467636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:01.527539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddb1ca-db29-4f02-8962-47b37531484f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:01.553122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:01.565371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:01.566019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.566299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.566571 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.567189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:01.568976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:01.572835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d042da-b926-4ba6-9223-6066b64b53c8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:01.575236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:01.579339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:01.580353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:01.581561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 497/977] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:23:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:01.586654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:01.614854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:01.615473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.615951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.616071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.616737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:01.617039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:01.618466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:01.640856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 481/978] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Aug 31 04:23:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:01.641884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:01.696071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466dd68-cffb-479f-bd34-a26c2f60640f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:01.729799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:01.730405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.730700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.730992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.731622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:01.731918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:01.733284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:01.738044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 498/979] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:23:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:01.739049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:01.742794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c63a1a-1fd8-436c-90af-ce74fa42a400 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:01.778696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70362a77-5c66-4aef-a88e-bc87913491c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:01.779320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70362a77-5c66-4aef-a88e-bc87913491c1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.779593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70362a77-5c66-4aef-a88e-bc87913491c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.779859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70362a77-5c66-4aef-a88e-bc87913491c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.792854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:01.812991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:01.841677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54739cb-2e15-492f-92f8-746963d2827b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:01.852831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70362a77-5c66-4aef-a88e-bc87913491c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:01.854280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70362a77-5c66-4aef-a88e-bc87913491c1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:01.874307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9d7bd-1a49-444f-af82-e6a2c7a52e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:01.874953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d7bd-1a49-444f-af82-e6a2c7a52e75 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:01.875232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d7bd-1a49-444f-af82-e6a2c7a52e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:01.875498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d7bd-1a49-444f-af82-e6a2c7a52e75 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:01.876153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9d7bd-1a49-444f-af82-e6a2c7a52e75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:01.877519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9d7bd-1a49-444f-af82-e6a2c7a52e75 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:01.893013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 499/980] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:23:01 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:23:01.893843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9d7bd-1a49-444f-af82-e6a2c7a52e75 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:01.966211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 482/981] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:01 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 04:23:01.967500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70362a77-5c66-4aef-a88e-bc87913491c1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:02.023590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9d7bd-1a49-444f-af82-e6a2c7a52e75 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:02.031096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:02.035124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:02.066996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:02.067622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:02.067903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:02.068178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:02.069291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:02.069594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:02.072066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:02.079981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 500/982] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:23:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:02.080557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70362a77-5c66-4aef-a88e-bc87913491c1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:02.086867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:02.119908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:02.120456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:02.120737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:02.121010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:02.132339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:02.133788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:02.151654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:02.152705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:02.153582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:02.153582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:02.153582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:02.154119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 483/983] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:02.159187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:02.211661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ab647-9b98-4e17-82a5-8ce8460fcb3c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:02.256858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:02.258035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:02.258507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:02.258979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:02.259418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c1b84-a4ac-4837-bcf8-48a0b3bc7136 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:02.260075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] RBAC: Adding query filter params (interface=admin, service_id=bf0aa09d380543ca9572fec3ed6f0ce3, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:02.260583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:02.263072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:02.274084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 501/984] 10.222.0.42 () {56 vars in 1249 bytes} [Wed Aug 31 04:23:02 2022] GET /identity/v3/endpoints?service_id=bf0aa09d380543ca9572fec3ed6f0ce3&interface=admin®ion_id=RegionOne => generated 193 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:02.274871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:02.278612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:02.302287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:02.302923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:02.303196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:02.303466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:02.304105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:02.304400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:02.305816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:02.311369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:02.318106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 484/985] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:02.323857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:02.381413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730bb7-ea8c-4931-a2a6-c06732dd075e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:02.416993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18638a2-9e23-47d3-b667-285101f02d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:02.417608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18638a2-9e23-47d3-b667-285101f02d40 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:02.417881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18638a2-9e23-47d3-b667-285101f02d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:02.418253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18638a2-9e23-47d3-b667-285101f02d40 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:02.419893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18638a2-9e23-47d3-b667-285101f02d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:02.421284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18638a2-9e23-47d3-b667-285101f02d40 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:02.427694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 502/986] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:23:02 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:23:02.444091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d6d3a-dfc6-44c0-b4dc-653bf07dd875 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:02.482674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:02.483362 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:02.483637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:02.483903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:02.491615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:02.493093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:23:02.506132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:02.507148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:02.508040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 485/987] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:02.516082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18638a2-9e23-47d3-b667-285101f02d40 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:02.620888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:02.624583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18638a2-9e23-47d3-b667-285101f02d40 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:02.658534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8936ef-15a0-4f06-a3d1-946305c30fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:02.659090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8936ef-15a0-4f06-a3d1-946305c30fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:02.659364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8936ef-15a0-4f06-a3d1-946305c30fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:02.659631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8936ef-15a0-4f06-a3d1-946305c30fa4 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:02.660320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8936ef-15a0-4f06-a3d1-946305c30fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:02.660620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8936ef-15a0-4f06-a3d1-946305c30fa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:02.662045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8936ef-15a0-4f06-a3d1-946305c30fa4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:02.674445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 503/988] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:23:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:02.680398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:02.716705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:02.786472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f21000-0b99-4d68-823b-def8bb3ca34c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:02.822399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef338f-bb6a-4372-a98f-dea1f3da7fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:02.823034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef338f-bb6a-4372-a98f-dea1f3da7fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:02.823308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef338f-bb6a-4372-a98f-dea1f3da7fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:02.823542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef338f-bb6a-4372-a98f-dea1f3da7fa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:02.824283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef338f-bb6a-4372-a98f-dea1f3da7fa5 None admin] RBAC: Adding query filter params (role.id=d15c78e6c27741619010a21e9a172da7, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=8381a7bfacbc41b39631c63b29359220) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:02.824632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef338f-bb6a-4372-a98f-dea1f3da7fa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:02.826131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef338f-bb6a-4372-a98f-dea1f3da7fa5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:02.834812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 486/989] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:02 2022] GET /identity/v3/role_assignments?user.id=8381a7bfacbc41b39631c63b29359220&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=d15c78e6c27741619010a21e9a172da7 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:03.844802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce8d2-d64e-4cca-b45d-8f265a09e145 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:03.845377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce8d2-d64e-4cca-b45d-8f265a09e145 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:03.845377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce8d2-d64e-4cca-b45d-8f265a09e145 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:03.846654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 504/990] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:03.852083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdce9fd-1cf2-4d72-a4ec-3edc832dc101 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:03.852528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdce9fd-1cf2-4d72-a4ec-3edc832dc101 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:03.852811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdce9fd-1cf2-4d72-a4ec-3edc832dc101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:03.871880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9cdce9fd-1cf2-4d72-a4ec-3edc832dc101 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:03.873108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9cdce9fd-1cf2-4d72-a4ec-3edc832dc101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:03.942838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 487/991] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:04.203830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cce8d2-d64e-4cca-b45d-8f265a09e145 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:04.311799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cce8d2-d64e-4cca-b45d-8f265a09e145 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:04.330851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356fc3f-b5e1-4fa6-a0e8-9fd94d64289f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.331136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356fc3f-b5e1-4fa6-a0e8-9fd94d64289f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.331405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356fc3f-b5e1-4fa6-a0e8-9fd94d64289f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.332760 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 488/992] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:04.338573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fc7b3-6ed4-4cc1-8941-dba4c0fc7f1d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.339299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fc7b3-6ed4-4cc1-8941-dba4c0fc7f1d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.340174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fc7b3-6ed4-4cc1-8941-dba4c0fc7f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.351237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:04.351895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.352172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.352440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.353030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:04.354401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:04.363283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a03fc7b3-6ed4-4cc1-8941-dba4c0fc7f1d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:04.363871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:04.364805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:23:04.365663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:04.366384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a03fc7b3-6ed4-4cc1-8941-dba4c0fc7f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:04.366464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 505/993] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:23:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:04.370826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:04.396592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:04.399347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:04.446312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 489/994] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:04.481984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f36d9-b0b5-45d6-a341-d5640416da80 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:04.517788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:04.518415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.518676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.518972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.519621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] RBAC: Adding query filter params (name=compute) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:04.519916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:04.521280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:04.526332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 506/995] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:23:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:04.532012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fc7b3-6ed4-4cc1-8941-dba4c0fc7f1d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:04.630840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fc7b3-6ed4-4cc1-8941-dba4c0fc7f1d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:04.664022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:04.664630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.664899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.665159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.665787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:04.666079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:04.667456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:04.672568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 490/996] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:23:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:04.678300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:04.719904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:04.745068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:04.772103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:04.788138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372cb13-9878-4da8-a55b-02dd396d70af None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:04.827659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580eef-68f7-4411-8ca1-43c038cc2d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:04.828288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580eef-68f7-4411-8ca1-43c038cc2d57 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.828580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580eef-68f7-4411-8ca1-43c038cc2d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.829081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580eef-68f7-4411-8ca1-43c038cc2d57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.830028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580eef-68f7-4411-8ca1-43c038cc2d57 None admin] RBAC: Adding query filter params (interface=public, service_id=9993394184ad472e84f1d2164bc3b205, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:04.830136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580eef-68f7-4411-8ca1-43c038cc2d57 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:04.831630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580eef-68f7-4411-8ca1-43c038cc2d57 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:04.832394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031ff93-0a3a-4156-9d59-e614ac2ccbe8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:04.841597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 507/997] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 31 04:23:04 2022] GET /identity/v3/endpoints?service_id=9993394184ad472e84f1d2164bc3b205&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:04.841597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77fe46-f9f5-41c5-bedb-7467e1edad8e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.841597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77fe46-f9f5-41c5-bedb-7467e1edad8e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.841597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77fe46-f9f5-41c5-bedb-7467e1edad8e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.841597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 508/998] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:04.842584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77fe46-f9f5-41c5-bedb-7467e1edad8e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:04.869971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfcc31-dcea-41b8-afbd-b0f9434e9146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:04.870612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfcc31-dcea-41b8-afbd-b0f9434e9146 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.870920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfcc31-dcea-41b8-afbd-b0f9434e9146 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.871192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfcc31-dcea-41b8-afbd-b0f9434e9146 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.872232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfcc31-dcea-41b8-afbd-b0f9434e9146 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:04.873625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfcc31-dcea-41b8-afbd-b0f9434e9146 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-55dfcc31-dcea-41b8-afbd-b0f9434e9146 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:04.882281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:04.883497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:04.884481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 491/999] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Aug 31 04:23:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:04.885449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6887380-3db5-44c7-ad5f-e7c3b1201723 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.885449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6887380-3db5-44c7-ad5f-e7c3b1201723 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.885449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6887380-3db5-44c7-ad5f-e7c3b1201723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.903776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6887380-3db5-44c7-ad5f-e7c3b1201723 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:04.904966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6887380-3db5-44c7-ad5f-e7c3b1201723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:04.945710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77fe46-f9f5-41c5-bedb-7467e1edad8e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:04.972469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 492/1000] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:04.973295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6887380-3db5-44c7-ad5f-e7c3b1201723 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:04.979888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b453ed-b856-46f1-8ce4-7398b90b2222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:04.980748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b453ed-b856-46f1-8ce4-7398b90b2222 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:04.981674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b453ed-b856-46f1-8ce4-7398b90b2222 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:04.982122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b453ed-b856-46f1-8ce4-7398b90b2222 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:04.982987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b453ed-b856-46f1-8ce4-7398b90b2222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:04.985135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b453ed-b856-46f1-8ce4-7398b90b2222 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:04.990704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:05.004163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 509/1001] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:23:04 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:23:05.088247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6887380-3db5-44c7-ad5f-e7c3b1201723 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:05.128313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:05.128936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:05.129207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:05.129475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:05.130112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:05.130408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:05.131823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:05.138758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 493/1002] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:23:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:05.143395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b453ed-b856-46f1-8ce4-7398b90b2222 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:05.162679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:05.237146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:05.247775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b453ed-b856-46f1-8ce4-7398b90b2222 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:05.283783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:05.284321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:05.284594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:05.284862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:05.285478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:05.285776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:05.287189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:05.292372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 510/1003] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Aug 31 04:23:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:05.297868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:05.327819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:05.348719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41d4fa-be96-4e9d-a56c-e7735da31829 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:05.384117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:05.384865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:05.385207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:05.385543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] PATH_INFO: `/v3/services/image` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:05.386004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:05.386193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:05.388088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:05.397460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:05.398454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:05.399303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 494/1004] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Aug 31 04:23:05 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:05.404574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:05.410670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be20c0-1715-49ac-bd04-2d09c513a66d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:05.447481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a1173-dc7d-4f67-8968-15ee956ed33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:05.448097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1173-dc7d-4f67-8968-15ee956ed33d None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:05.448374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1173-dc7d-4f67-8968-15ee956ed33d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:05.448642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1173-dc7d-4f67-8968-15ee956ed33d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:05.449301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a1173-dc7d-4f67-8968-15ee956ed33d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:05.450706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a1173-dc7d-4f67-8968-15ee956ed33d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:05.470272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 511/1005] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:23:05 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:23:05.515235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cc054-94dc-48e7-b9d2-1a5b49acb198 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:05.561334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcdbff-5a2f-4197-92c1-86835e1626f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:05.561855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcdbff-5a2f-4197-92c1-86835e1626f6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:05.562210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcdbff-5a2f-4197-92c1-86835e1626f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:05.562385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcdbff-5a2f-4197-92c1-86835e1626f6 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:05.563086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcdbff-5a2f-4197-92c1-86835e1626f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:05.563379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcdbff-5a2f-4197-92c1-86835e1626f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:05.564765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcdbff-5a2f-4197-92c1-86835e1626f6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:05.571062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 495/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:05.576894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a1173-dc7d-4f67-8968-15ee956ed33d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:05.687776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a1173-dc7d-4f67-8968-15ee956ed33d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:05.725457 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3299d94-40bd-4285-80d1-5e1666ebee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:05.726446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3299d94-40bd-4285-80d1-5e1666ebee22 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:05.726974 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3299d94-40bd-4285-80d1-5e1666ebee22 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:05.727447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3299d94-40bd-4285-80d1-5e1666ebee22 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:05.728280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3299d94-40bd-4285-80d1-5e1666ebee22 None admin] RBAC: Adding query filter params (type=image) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:05.728790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3299d94-40bd-4285-80d1-5e1666ebee22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:05.730356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3299d94-40bd-4285-80d1-5e1666ebee22 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:05.736458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 512/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:06.918001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b7811-7419-4281-a280-c17876629320 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:06.919177 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b7811-7419-4281-a280-c17876629320 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:06.919177 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b7811-7419-4281-a280-c17876629320 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:06.920576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 496/1008] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:06.925842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14ddfb-8e23-4de0-b30e-6eaf2055064d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:06.926245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14ddfb-8e23-4de0-b30e-6eaf2055064d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:06.926431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14ddfb-8e23-4de0-b30e-6eaf2055064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:06.933608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:06.946507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9f14ddfb-8e23-4de0-b30e-6eaf2055064d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:06.947738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9f14ddfb-8e23-4de0-b30e-6eaf2055064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:07.019402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 513/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:07.280387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b7811-7419-4281-a280-c17876629320 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:07.284392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:07.384572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10beae-7c9f-4852-85db-6ae831e8313f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.384805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b7811-7419-4281-a280-c17876629320 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:07.384892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10beae-7c9f-4852-85db-6ae831e8313f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.385125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10beae-7c9f-4852-85db-6ae831e8313f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.386528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 514/1010] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:07.392908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde103ed-e2ad-493a-a528-4666f98823a9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.393204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde103ed-e2ad-493a-a528-4666f98823a9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.393461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde103ed-e2ad-493a-a528-4666f98823a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.413424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cde103ed-e2ad-493a-a528-4666f98823a9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:07.414615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cde103ed-e2ad-493a-a528-4666f98823a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:07.420769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:07.421402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.421676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.421942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.422559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:07.423928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:07.432663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:07.433839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:07.434703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 497/1011] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:23:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:07.439598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:07.487763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 515/1012] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:07 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:23:07.515517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:07.550691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051b445-8ebf-4b5f-8a6c-cb5c203fe06f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:07.585565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86099871-8e6d-48ea-b9e0-771cffb961ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:07.586168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86099871-8e6d-48ea-b9e0-771cffb961ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.586437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86099871-8e6d-48ea-b9e0-771cffb961ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.586703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86099871-8e6d-48ea-b9e0-771cffb961ff None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.587369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86099871-8e6d-48ea-b9e0-771cffb961ff None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:07.587663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86099871-8e6d-48ea-b9e0-771cffb961ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:07.589019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86099871-8e6d-48ea-b9e0-771cffb961ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:07.594053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 498/1013] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:23:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:07.599655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde103ed-e2ad-493a-a528-4666f98823a9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:07.629591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdedfc-dec7-4805-b5b4-e0a65a6773da None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.629903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdedfc-dec7-4805-b5b4-e0a65a6773da None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.630132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdedfc-dec7-4805-b5b4-e0a65a6773da None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.631495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 499/1014] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:23:07.636257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f397c6-a51e-4683-a9db-98c86c2a7608 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.636533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f397c6-a51e-4683-a9db-98c86c2a7608 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.636800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f397c6-a51e-4683-a9db-98c86c2a7608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.654674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-59f397c6-a51e-4683-a9db-98c86c2a7608 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:07.655962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-59f397c6-a51e-4683-a9db-98c86c2a7608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:07.667950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:07.713518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde103ed-e2ad-493a-a528-4666f98823a9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:07.725536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 500/1015] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:07 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:23:07.752983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:07.753605 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.753816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f397c6-a51e-4683-a9db-98c86c2a7608 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:07.753919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.754139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.754802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:07.755101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:07.756570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:07.761707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 516/1016] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Aug 31 04:23:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:07.767722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:07.819654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:07.865606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f397c6-a51e-4683-a9db-98c86c2a7608 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:07.869826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec02840-7547-48c1-b21d-3210ee1f4db7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:07.897569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:07.901464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e58978-e3a2-411d-b313-a7c28536a98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:07.902104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e58978-e3a2-411d-b313-a7c28536a98c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.902378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e58978-e3a2-411d-b313-a7c28536a98c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.902658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e58978-e3a2-411d-b313-a7c28536a98c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.903979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e58978-e3a2-411d-b313-a7c28536a98c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:07.905436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e58978-e3a2-411d-b313-a7c28536a98c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:07.906916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d5e87-ab3d-4870-acab-8b57f673682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:07.907529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d5e87-ab3d-4870-acab-8b57f673682f None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:07.907810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d5e87-ab3d-4870-acab-8b57f673682f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:07.908082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d5e87-ab3d-4870-acab-8b57f673682f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:07.908751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d5e87-ab3d-4870-acab-8b57f673682f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:07.910122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d5e87-ab3d-4870-acab-8b57f673682f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:07.911979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 501/1017] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:07.912781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e58978-e3a2-411d-b313-a7c28536a98c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:07.933643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 517/1018] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:23:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:23:07.995039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d5e87-ab3d-4870-acab-8b57f673682f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:08.024170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e58978-e3a2-411d-b313-a7c28536a98c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:08.055439 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:08.072688 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01f954-2f28-4c74-9333-2a93da2639f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:08.073314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01f954-2f28-4c74-9333-2a93da2639f7 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:08.073589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01f954-2f28-4c74-9333-2a93da2639f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:08.073856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01f954-2f28-4c74-9333-2a93da2639f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:08.074598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01f954-2f28-4c74-9333-2a93da2639f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:08.076878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01f954-2f28-4c74-9333-2a93da2639f7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:08.110583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 502/1019] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:23:07 2022] POST /identity/v3/projects => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:23:08.119331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d5e87-ab3d-4870-acab-8b57f673682f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:08.161807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6302de-1b75-47cd-ba73-141577a5fc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:08.162446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6302de-1b75-47cd-ba73-141577a5fc52 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:08.162832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6302de-1b75-47cd-ba73-141577a5fc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:08.163104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6302de-1b75-47cd-ba73-141577a5fc52 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:08.163792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6302de-1b75-47cd-ba73-141577a5fc52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:08.165282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6302de-1b75-47cd-ba73-141577a5fc52 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:08.183543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 518/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:23:07 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:23:09.954648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7ea91-540e-41f2-b3a0-2f5f56846a6e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:09.955252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7ea91-540e-41f2-b3a0-2f5f56846a6e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:09.955252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7ea91-540e-41f2-b3a0-2f5f56846a6e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:09.956714 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 503/1021] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:09.962047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b366d-c77a-4739-a747-c0536db4c713 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:09.962511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b366d-c77a-4739-a747-c0536db4c713 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:09.962589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b366d-c77a-4739-a747-c0536db4c713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:09.982896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ed7b366d-c77a-4739-a747-c0536db4c713 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:09.984091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ed7b366d-c77a-4739-a747-c0536db4c713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:10.027035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:10.072064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 519/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:10.107308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9cc9-448e-40ae-88b2-92a21beb88d4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.107621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9cc9-448e-40ae-88b2-92a21beb88d4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.107862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9cc9-448e-40ae-88b2-92a21beb88d4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.109256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 504/1023] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:10.114604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624a6fa-846d-4f07-9c66-38dd5e7d544a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.114897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624a6fa-846d-4f07-9c66-38dd5e7d544a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.115190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624a6fa-846d-4f07-9c66-38dd5e7d544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.125559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f462ba-a484-4712-9157-05f0fe1cd512 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.125830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f462ba-a484-4712-9157-05f0fe1cd512 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.126104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f462ba-a484-4712-9157-05f0fe1cd512 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.127408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 505/1024] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:23:10.132553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa680fd7-b116-493f-a35e-b02d2a5341b6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.132847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa680fd7-b116-493f-a35e-b02d2a5341b6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.133120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa680fd7-b116-493f-a35e-b02d2a5341b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.134991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b624a6fa-846d-4f07-9c66-38dd5e7d544a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:10.136196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b624a6fa-846d-4f07-9c66-38dd5e7d544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:10.152244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-aa680fd7-b116-493f-a35e-b02d2a5341b6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:10.153927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aa680fd7-b116-493f-a35e-b02d2a5341b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:10.205441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 520/1025] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:10.228451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 506/1026] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:10.335369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624a6fa-846d-4f07-9c66-38dd5e7d544a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:10.436584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:10.440550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624a6fa-846d-4f07-9c66-38dd5e7d544a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:10.468047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa680fd7-b116-493f-a35e-b02d2a5341b6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:10.477078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1643b68-5853-4eda-a309-8ac3bb2c94eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:10.477718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1643b68-5853-4eda-a309-8ac3bb2c94eb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.477996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1643b68-5853-4eda-a309-8ac3bb2c94eb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.478266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1643b68-5853-4eda-a309-8ac3bb2c94eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.488931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1643b68-5853-4eda-a309-8ac3bb2c94eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e1643b68-5853-4eda-a309-8ac3bb2c94eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:10.494173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:10.495241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:10.496076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 521/1027] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 31 04:23:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:10.501879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1643b68-5853-4eda-a309-8ac3bb2c94eb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:10.571001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa680fd7-b116-493f-a35e-b02d2a5341b6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:10.581069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:10.605694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1643b68-5853-4eda-a309-8ac3bb2c94eb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:10.608002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0c9c8-fd9f-4ef8-b8ae-2c425cb88263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:10.608637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0c9c8-fd9f-4ef8-b8ae-2c425cb88263 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.608909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0c9c8-fd9f-4ef8-b8ae-2c425cb88263 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.609173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0c9c8-fd9f-4ef8-b8ae-2c425cb88263 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.610274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0c9c8-fd9f-4ef8-b8ae-2c425cb88263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:10.612051 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0c9c8-fd9f-4ef8-b8ae-2c425cb88263 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:10.613786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 507/1028] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:10.615631 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0c9c8-fd9f-4ef8-b8ae-2c425cb88263 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:10.643381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:10.644031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.644310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.644584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.645263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:10.645568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:10.647007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:10.655839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 522/1029] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 31 04:23:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:10.656638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:10.681686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:10.722087 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0c9c8-fd9f-4ef8-b8ae-2c425cb88263 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:10.755409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:10.756040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.756151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bad20-1a05-47d1-8e05-231b0c7a86e7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:10.756316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.756590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.757178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:10.758559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:10.767615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:10.768775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:10.769763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 508/1030] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Aug 31 04:23:10 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:10.770762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:10.795521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb622f48-2668-442f-88bf-cecdabbf5186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:10.796149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622f48-2668-442f-88bf-cecdabbf5186 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.796424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622f48-2668-442f-88bf-cecdabbf5186 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.796693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb622f48-2668-442f-88bf-cecdabbf5186 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.797341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb622f48-2668-442f-88bf-cecdabbf5186 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:10.798803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb622f48-2668-442f-88bf-cecdabbf5186 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:10.811000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:10.822098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 523/1031] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:23:10 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:23:10.826942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb622f48-2668-442f-88bf-cecdabbf5186 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:10.878883 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280c1f-aa6e-4b82-8c2d-904dd7c48de8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:10.912627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:10.922641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:10.923324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.923598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.923862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.936982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:10.937197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb622f48-2668-442f-88bf-cecdabbf5186 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:10.938484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:10.956134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:10.957646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:10.958534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:10.958534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:10.958534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:10.958534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 509/1032] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:10.963508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:10.984498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:10.985046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:10.985326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:10.985590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:10.986289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:10.987220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:10.988603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:10.994436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 524/1033] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:11.000800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:11.074278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:11.083188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8a10a-31c3-46a7-ace5-e229756dc742 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:11.106451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe095dff-5400-4b41-a183-b82bd5e24de5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:11.118197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:11.118839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:11.119114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:11.119380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:11.120034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:11.120326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:11.121718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:11.134777 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 510/1034] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:11.140543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:11.146820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:11.147436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:11.147712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:11.147977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:11.148601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] RBAC: Adding query filter params (type=image) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:11.148893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:11.150250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:11.155407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 525/1035] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:11.161082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:11.170758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:11.245707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04715b18-4639-4c7d-9146-4b86fed8ecc7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:11.266002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498b40b-bf5b-4ba5-b160-412969e00551 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:11.283316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:11.284289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:11.284783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:11.285266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:11.292903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:11.294651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:11.301261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:11.301861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:11.302114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:11.302383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:11.303116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] RBAC: Adding query filter params (interface=public, service_id=7b986ba5a5de4ce4b8ba5bfd634aa68c, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:11.303419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:11.304781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:23:11.308093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:11.309150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:23:11.310284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:11.310862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 526/1036] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 31 04:23:11 2022] GET /identity/v3/endpoints?service_id=7b986ba5a5de4ce4b8ba5bfd634aa68c&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:11.312285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 511/1037] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:23:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:11.315866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:11.317797 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:11.338266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:11.391295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:11.425327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31af02-9b4d-4cae-b548-1fcf8a0f7e32 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:11.434576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4c30-0ab4-443d-9cd8-737527095ca8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:11.461693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a20c-3b3d-4eca-963b-1cc1a45c27c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:11.462307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a20c-3b3d-4eca-963b-1cc1a45c27c2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:11.462584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a20c-3b3d-4eca-963b-1cc1a45c27c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:11.462883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a20c-3b3d-4eca-963b-1cc1a45c27c2 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:11.463489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6a20c-3b3d-4eca-963b-1cc1a45c27c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:11.464910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6a20c-3b3d-4eca-963b-1cc1a45c27c2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:11.468743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:11.469350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:11.469626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:11.469892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:11.470615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:11.470939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:11.471721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 527/1038] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:23:11 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:23:11.472441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:11.497628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 512/1039] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:23:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:11.498813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a20c-3b3d-4eca-963b-1cc1a45c27c2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:11.610433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a20c-3b3d-4eca-963b-1cc1a45c27c2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:11.646947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05e5ff-f5ab-4a73-82a2-83cd89276a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:11.647572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e5ff-f5ab-4a73-82a2-83cd89276a22 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:11.647862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e5ff-f5ab-4a73-82a2-83cd89276a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:11.648125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e5ff-f5ab-4a73-82a2-83cd89276a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:11.650829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:11.717012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e5ff-f5ab-4a73-82a2-83cd89276a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:11.718500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e5ff-f5ab-4a73-82a2-83cd89276a22 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:11.820756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 528/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:11.827032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:11.898048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:11.930162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972b49a-8ab8-449e-a79b-77b5f03ea46d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:11.972072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:11.972732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:11.972969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:11.973236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:11.983963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:11.985419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:12.000436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:12.001553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:12.002454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:12.002454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:12.002454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:12.006768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 513/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:12.007480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05e5ff-f5ab-4a73-82a2-83cd89276a22 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:12.041745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:12.120356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05e5ff-f5ab-4a73-82a2-83cd89276a22 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:12.155520 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:12.156128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:12.156401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:12.156671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:12.157309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:12.157607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:12.159001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:12.170500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 529/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:12.176051 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:12.275867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0591278-bab3-4006-bfb3-94ccfd3824ca None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:12.304593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:12.317794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:12.318435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:12.318794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:12.319420 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:12.326331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:12.327835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:23:12.340788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:12.341807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:12.342670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:23:12.342670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:23:12.342670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:23:12.342670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:23:12.342670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:23:12.342670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:23:12.342670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:23:12.342670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:12.347092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 514/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:12.348528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:12.449764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:12.457658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae27d-bd4a-41a4-b747-36c0049b820a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:12.491522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a799670-176f-4fa7-a4d9-332f0da662b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:12.492134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799670-176f-4fa7-a4d9-332f0da662b0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:12.492407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799670-176f-4fa7-a4d9-332f0da662b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:12.492667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799670-176f-4fa7-a4d9-332f0da662b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:12.493385 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a799670-176f-4fa7-a4d9-332f0da662b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:12.493686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a799670-176f-4fa7-a4d9-332f0da662b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:12.495155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a799670-176f-4fa7-a4d9-332f0da662b0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:12.507221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 530/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:12.513511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:12.623517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b378d9-f949-4fcd-ae2b-1a614ce1def9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:12.660036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3301454-765b-4d57-9edf-54292ce75707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:12.660648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3301454-765b-4d57-9edf-54292ce75707 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:12.660923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3301454-765b-4d57-9edf-54292ce75707 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:12.661212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3301454-765b-4d57-9edf-54292ce75707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:12.661927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3301454-765b-4d57-9edf-54292ce75707 None admin] RBAC: Adding query filter params (role.id=d15c78e6c27741619010a21e9a172da7, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=708ce6cfc5f64d26a418b4339fd883fb) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:12.662233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3301454-765b-4d57-9edf-54292ce75707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:12.663697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3301454-765b-4d57-9edf-54292ce75707 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:12.675088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 515/1045] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:12 2022] GET /identity/v3/role_assignments?user.id=708ce6cfc5f64d26a418b4339fd883fb&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=d15c78e6c27741619010a21e9a172da7 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:12.776629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93f1e-2692-4624-a189-1345039f2624 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:12.776902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93f1e-2692-4624-a189-1345039f2624 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:12.777130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93f1e-2692-4624-a189-1345039f2624 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:12.778778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 531/1046] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:12.784780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ac35-886b-4d04-ba4b-579942b301e4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:12.785103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ac35-886b-4d04-ba4b-579942b301e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:12.785297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ac35-886b-4d04-ba4b-579942b301e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:12.805866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9758ac35-886b-4d04-ba4b-579942b301e4 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:12.807282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9758ac35-886b-4d04-ba4b-579942b301e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:12.853656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:12.882801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 516/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:13.144086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c93f1e-2692-4624-a189-1345039f2624 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:13.255216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c93f1e-2692-4624-a189-1345039f2624 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:13.289885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed095301-a46f-405b-8cc5-b11db27ee937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:13.290527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed095301-a46f-405b-8cc5-b11db27ee937 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:13.290838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed095301-a46f-405b-8cc5-b11db27ee937 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:13.291108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed095301-a46f-405b-8cc5-b11db27ee937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:13.301842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed095301-a46f-405b-8cc5-b11db27ee937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ed095301-a46f-405b-8cc5-b11db27ee937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:13.306625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:13.307637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:13.311695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 532/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:23:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:13.313997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758ac35-886b-4d04-ba4b-579942b301e4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:13.385717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8072bb-9f24-4321-b13f-19b788c5221a None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:13.385898 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8072bb-9f24-4321-b13f-19b788c5221a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:13.386185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8072bb-9f24-4321-b13f-19b788c5221a None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:13.387779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 533/1049] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:23:13.392651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e899bc-a55f-462c-a7be-8d9a83f977b5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:13.393089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e899bc-a55f-462c-a7be-8d9a83f977b5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:13.393360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e899bc-a55f-462c-a7be-8d9a83f977b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:13.411613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-40e899bc-a55f-462c-a7be-8d9a83f977b5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:13.412799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-40e899bc-a55f-462c-a7be-8d9a83f977b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:13.419882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758ac35-886b-4d04-ba4b-579942b301e4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:13.422418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:13.457084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:13.458046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:13.458325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:13.458591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:13.459599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:13.460056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:13.461422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:13.470111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 517/1050] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:23:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:13.475946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:13.484620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 534/1051] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:23:13.516225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:13.578689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5f2a0-57f6-4748-b9ab-087a7504476c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:13.612430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271cf37-9f59-4559-98e2-3c1914302676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:13.613054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271cf37-9f59-4559-98e2-3c1914302676 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:13.613332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271cf37-9f59-4559-98e2-3c1914302676 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:13.613598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271cf37-9f59-4559-98e2-3c1914302676 None admin] PATH_INFO: `/v3/users/9659619c4be34dd799f784405f31e062` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:13.627780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271cf37-9f59-4559-98e2-3c1914302676 None admin] RBAC: Authorizing `identity:get_user(user_id=9659619c4be34dd799f784405f31e062)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:13.629220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271cf37-9f59-4559-98e2-3c1914302676 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:13.631062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 518/1052] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:13 2022] GET /identity/v3/users/9659619c4be34dd799f784405f31e062 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:13.637052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e899bc-a55f-462c-a7be-8d9a83f977b5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:13.738959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e899bc-a55f-462c-a7be-8d9a83f977b5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:13.747861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271cf37-9f59-4559-98e2-3c1914302676 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:13.779186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbbba2-0892-493a-9ba4-74c847894e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:13.779910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbbba2-0892-493a-9ba4-74c847894e5b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:13.780193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbbba2-0892-493a-9ba4-74c847894e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:13.780579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbbba2-0892-493a-9ba4-74c847894e5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:13.796434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:13.847920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:13.855181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271cf37-9f59-4559-98e2-3c1914302676 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:13.856763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbbba2-0892-493a-9ba4-74c847894e5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:13.858540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbbba2-0892-493a-9ba4-74c847894e5b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:13.892392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:13.892951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:13.893268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:13.893638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:13.894229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:13.896042 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:13.905802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:13.906824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:13.908235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 519/1053] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Aug 31 04:23:13 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:23:13.912727 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:13.961377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 535/1054] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:13 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:13.967825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbbba2-0892-493a-9ba4-74c847894e5b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:14.019953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:14.031824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b30a33-a32f-417c-ac67-ade2a53df2f8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:14.079125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:14.079742 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:14.080017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:14.080288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:14.080927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:14.081224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:14.082604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:14.087719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 520/1055] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:14.093370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:14.097350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbbba2-0892-493a-9ba4-74c847894e5b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:14.118633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:14.134117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a6c13-ee25-415a-a82d-96edba1e5323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:14.135177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a6c13-ee25-415a-a82d-96edba1e5323 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:14.135471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a6c13-ee25-415a-a82d-96edba1e5323 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:14.135748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a6c13-ee25-415a-a82d-96edba1e5323 None admin] PATH_INFO: `/v3/projects/83b467d008eb42c1b06fd8964ce90adc` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:14.137616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a6c13-ee25-415a-a82d-96edba1e5323 None admin] RBAC: Authorizing `identity:get_project(project_id=83b467d008eb42c1b06fd8964ce90adc)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:14.139577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a6c13-ee25-415a-a82d-96edba1e5323 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:14.146148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 536/1056] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:13 2022] GET /identity/v3/projects/83b467d008eb42c1b06fd8964ce90adc => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:14.148040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a6c13-ee25-415a-a82d-96edba1e5323 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:14.204172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5815e-a966-4449-ae90-08cc78576a03 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:14.241107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:14.241715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:14.241996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:14.242247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:14.242909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] RBAC: Adding query filter params (type=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:14.243205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:14.244565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:14.249765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 521/1057] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:14.255510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:14.255666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a6c13-ee25-415a-a82d-96edba1e5323 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:14.271900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:14.293767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1b9d4-f601-4346-895b-47c5a21f786f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:14.294448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1b9d4-f601-4346-895b-47c5a21f786f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:14.294748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1b9d4-f601-4346-895b-47c5a21f786f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:14.295012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1b9d4-f601-4346-895b-47c5a21f786f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:14.295732 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1b9d4-f601-4346-895b-47c5a21f786f None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=83b467d008eb42c1b06fd8964ce90adc, user.id=9659619c4be34dd799f784405f31e062) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:14.296093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1b9d4-f601-4346-895b-47c5a21f786f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:14.297521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1b9d4-f601-4346-895b-47c5a21f786f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:14.314762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 537/1058] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:14 2022] GET /identity/v3/role_assignments?user.id=9659619c4be34dd799f784405f31e062&scope.project.id=83b467d008eb42c1b06fd8964ce90adc&role.id=241e99d365564c0fa95cb712f4004785 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:14.365452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:14.385447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437febe5-b86c-406c-9406-599a69d6a260 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:14.424462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d95885-20ba-4e90-8274-1b7d858fa9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:14.425006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d95885-20ba-4e90-8274-1b7d858fa9df None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:14.425287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d95885-20ba-4e90-8274-1b7d858fa9df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:14.425511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d95885-20ba-4e90-8274-1b7d858fa9df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:14.426234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d95885-20ba-4e90-8274-1b7d858fa9df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:14.427658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d95885-20ba-4e90-8274-1b7d858fa9df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:14.452257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 522/1059] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:23:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:23:14.573258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc158bc-2469-46ba-a639-825f2c5bb14d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:14.573516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc158bc-2469-46ba-a639-825f2c5bb14d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:14.573789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc158bc-2469-46ba-a639-825f2c5bb14d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:14.575469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 538/1060] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:14.581182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6baaca-b864-41d3-a3c1-bfea7660a9ba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:14.581489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6baaca-b864-41d3-a3c1-bfea7660a9ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:14.581730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6baaca-b864-41d3-a3c1-bfea7660a9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:14.601552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9e6baaca-b864-41d3-a3c1-bfea7660a9ba None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:14.602755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9e6baaca-b864-41d3-a3c1-bfea7660a9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:14.680237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 523/1061] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:23:14.940486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc158bc-2469-46ba-a639-825f2c5bb14d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:15.046058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc158bc-2469-46ba-a639-825f2c5bb14d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:15.089133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6deed8-496c-4c8d-aeca-293e75ade828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:15.089763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6deed8-496c-4c8d-aeca-293e75ade828 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:15.090035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6deed8-496c-4c8d-aeca-293e75ade828 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:15.090299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6deed8-496c-4c8d-aeca-293e75ade828 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:15.100558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6deed8-496c-4c8d-aeca-293e75ade828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-5f6deed8-496c-4c8d-aeca-293e75ade828 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:15.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:15.106273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:15.111569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 539/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 31 04:23:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:15.113356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6baaca-b864-41d3-a3c1-bfea7660a9ba None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:15.186588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:15.218985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6baaca-b864-41d3-a3c1-bfea7660a9ba None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:15.252804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:15.253415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:15.253686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:15.253949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:15.254611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:15.254944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:15.256305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:15.264523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 524/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 31 04:23:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:15.270134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6deed8-496c-4c8d-aeca-293e75ade828 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:15.307819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:15.378658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6deed8-496c-4c8d-aeca-293e75ade828 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:15.416557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:15.417204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:15.417479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:15.417745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:15.429598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:15.431082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:15.447110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:15.448061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:15.448932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:15.448932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:15.448932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:15.448932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 540/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:15.455205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:15.558711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f4f5d-9968-4f09-a9b3-ad518ac110d3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:15.588525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:15.594594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:15.595219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:15.595498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:15.595763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:15.596406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:15.596701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:15.598096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:15.610935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 525/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:15.616490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:15.704833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:15.718477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7616e3-9315-4197-b9ec-07b176722249 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:15.751954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:15.752595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:15.752869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:15.753135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:15.760658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:15.762106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:23:15.773084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:15.774013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:23:15.775367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:15.780481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 541/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:23:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:15.780606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:15.881357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f8354-a6d3-4921-b2c7-5393ba1b2d07 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:15.915501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:15.916109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:15.916375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:15.916638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:15.917360 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:15.917657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:15.919157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:15.936479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 526/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:23:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:15.942050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.047650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93cda1-a5e8-407d-afb9-f386bf95555c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:16.088179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdac38b-8165-40b3-99f7-d7e856376af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:16.088825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdac38b-8165-40b3-99f7-d7e856376af7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.089071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdac38b-8165-40b3-99f7-d7e856376af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.089366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdac38b-8165-40b3-99f7-d7e856376af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.100948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdac38b-8165-40b3-99f7-d7e856376af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:16.102402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdac38b-8165-40b3-99f7-d7e856376af7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:16.111209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-dbdac38b-8165-40b3-99f7-d7e856376af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:16.117627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:16.119230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:16.120515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:16.120515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:16.120515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:16.123065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 542/1068] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:16.124833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.178584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:16.189441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f525fd-ac0d-4b0a-a7ac-5bc016d3de96 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.190053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f525fd-ac0d-4b0a-a7ac-5bc016d3de96 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.190471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f525fd-ac0d-4b0a-a7ac-5bc016d3de96 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.193126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 543/1069] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:16.198341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3202b6-2007-4c36-8142-083f95dee92c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.198690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3202b6-2007-4c36-8142-083f95dee92c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.199059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3202b6-2007-4c36-8142-083f95dee92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.218070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3c3202b6-2007-4c36-8142-083f95dee92c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:16.219382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3c3202b6-2007-4c36-8142-083f95dee92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:16.231441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fc026-aabf-405e-ac69-b608202720c9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:16.267028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:16.267730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.268007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.268305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.268953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:16.269318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:16.270741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:16.283113 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 527/1070] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:16.288678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.292023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 544/1071] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:23:16.394215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4afaef-7330-4fe5-8611-72a10286fd7c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:16.405061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:16.412224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b818-1a7b-4d2e-871d-c7a94384a070 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.412561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b818-1a7b-4d2e-871d-c7a94384a070 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.412774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b818-1a7b-4d2e-871d-c7a94384a070 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.414267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 545/1072] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:16.419318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26a522-dffc-4d12-b65f-a6f932c0e191 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.419639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26a522-dffc-4d12-b65f-a6f932c0e191 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.419834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26a522-dffc-4d12-b65f-a6f932c0e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.431251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:16.431813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.432086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.432616 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.438521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ac26a522-dffc-4d12-b65f-a6f932c0e191 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:16.439716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:16.439827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ac26a522-dffc-4d12-b65f-a6f932c0e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:16.441158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:23:16.453955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:16.454892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:16.456248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 528/1073] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:23:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:16.461050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.510211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 546/1074] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:16.555800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26a522-dffc-4d12-b65f-a6f932c0e191 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.566036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b1362-4557-48a7-8158-2d049c70f091 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:16.603185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:16.603796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.604076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.604349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.605063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:16.605370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:16.606923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:16.619718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 529/1075] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:16.619878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:16.625766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.657594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:16.667208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26a522-dffc-4d12-b65f-a6f932c0e191 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:16.703487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a762330-0907-4954-bb65-2aa82fcf1bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:16.704047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a762330-0907-4954-bb65-2aa82fcf1bac None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.704326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a762330-0907-4954-bb65-2aa82fcf1bac None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.704598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a762330-0907-4954-bb65-2aa82fcf1bac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.715181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a762330-0907-4954-bb65-2aa82fcf1bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8a762330-0907-4954-bb65-2aa82fcf1bac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:16.720264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:16.721173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:16.722591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 547/1076] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:16.727363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a762330-0907-4954-bb65-2aa82fcf1bac None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.736073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74c9a-c5d3-4fd4-a7de-c501d3a1904f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:16.773912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64578bf7-e0d5-422d-83fd-85185cbe37e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:16.774561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64578bf7-e0d5-422d-83fd-85185cbe37e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.774872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64578bf7-e0d5-422d-83fd-85185cbe37e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.775140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64578bf7-e0d5-422d-83fd-85185cbe37e3 None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/708ce6cfc5f64d26a418b4339fd883fb/roles/d15c78e6c27741619010a21e9a172da7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.777272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64578bf7-e0d5-422d-83fd-85185cbe37e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=708ce6cfc5f64d26a418b4339fd883fb, role_id=d15c78e6c27741619010a21e9a172da7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:16.779067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64578bf7-e0d5-422d-83fd-85185cbe37e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:16.792609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 530/1077] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:23:16 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/708ce6cfc5f64d26a418b4339fd883fb/roles/d15c78e6c27741619010a21e9a172da7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:23:16.793389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64578bf7-e0d5-422d-83fd-85185cbe37e3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.842242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a762330-0907-4954-bb65-2aa82fcf1bac None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:16.864902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:16.882759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:16.883400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.883663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.883930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:16.884599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:16.884898 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:16.886305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:16.898594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:16.901315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 548/1078] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:16.902328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:16.915174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64578bf7-e0d5-422d-83fd-85185cbe37e3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:16.955983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8766a4-1a7d-4680-abb2-35913671db3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:16.956516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8766a4-1a7d-4680-abb2-35913671db3e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:16.956788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8766a4-1a7d-4680-abb2-35913671db3e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:16.957053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8766a4-1a7d-4680-abb2-35913671db3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:17.015812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bedc7c-ef57-489c-a715-c8327e8c25f8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:17.028083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8766a4-1a7d-4680-abb2-35913671db3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:17.030122 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8766a4-1a7d-4680-abb2-35913671db3e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:17.050083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8bd2a-d21b-4ad3-bae6-ec51a67172a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:17.050742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bd2a-d21b-4ad3-bae6-ec51a67172a3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:17.051020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bd2a-d21b-4ad3-bae6-ec51a67172a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:17.051286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bd2a-d21b-4ad3-bae6-ec51a67172a3 None admin] PATH_INFO: `/v3/users/9659619c4be34dd799f784405f31e062` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:17.052440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8bd2a-d21b-4ad3-bae6-ec51a67172a3 None admin] RBAC: Authorizing `identity:get_user(user_id=9659619c4be34dd799f784405f31e062)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:17.053873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8bd2a-d21b-4ad3-bae6-ec51a67172a3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:17.055509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 549/1079] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:16 2022] GET /identity/v3/users/9659619c4be34dd799f784405f31e062 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:17.061428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8bd2a-d21b-4ad3-bae6-ec51a67172a3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:17.123780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:17.130651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:17.142813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 531/1080] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:17.148396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8766a4-1a7d-4680-abb2-35913671db3e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:17.176719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8bd2a-d21b-4ad3-bae6-ec51a67172a3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:17.212023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99fae-83ff-41e9-b6fe-f49e8c1023e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:17.212604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99fae-83ff-41e9-b6fe-f49e8c1023e0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:17.212869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99fae-83ff-41e9-b6fe-f49e8c1023e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:17.213121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99fae-83ff-41e9-b6fe-f49e8c1023e0 None admin] PATH_INFO: `/v3/projects/83b467d008eb42c1b06fd8964ce90adc` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:17.214265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f99fae-83ff-41e9-b6fe-f49e8c1023e0 None admin] RBAC: Authorizing `identity:get_project(project_id=83b467d008eb42c1b06fd8964ce90adc)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:17.215845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f99fae-83ff-41e9-b6fe-f49e8c1023e0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:17.222483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 550/1081] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:17 2022] GET /identity/v3/projects/83b467d008eb42c1b06fd8964ce90adc => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:17.223441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99fae-83ff-41e9-b6fe-f49e8c1023e0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:17.259568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8766a4-1a7d-4680-abb2-35913671db3e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:17.297372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:17.298020 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:17.298296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:17.298563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:17.310287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:17.311825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:17.327750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:17.328737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:17.329594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:17.329594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:17.329594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:17.329594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 532/1082] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:23:17.339584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:17.348557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99fae-83ff-41e9-b6fe-f49e8c1023e0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:17.367245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:17.384488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:17.389581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91d98d-25cb-47fc-b291-d8771a92ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:17.390925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91d98d-25cb-47fc-b291-d8771a92ca33 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:17.391206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91d98d-25cb-47fc-b291-d8771a92ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:17.391578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91d98d-25cb-47fc-b291-d8771a92ca33 None admin] PATH_INFO: `/v3/projects/83b467d008eb42c1b06fd8964ce90adc/users/9659619c4be34dd799f784405f31e062/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:17.393689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91d98d-25cb-47fc-b291-d8771a92ca33 None admin] RBAC: Authorizing `identity:create_grant(project_id=83b467d008eb42c1b06fd8964ce90adc, user_id=9659619c4be34dd799f784405f31e062, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:17.396336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91d98d-25cb-47fc-b291-d8771a92ca33 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:17.412518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 551/1083] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:23:17 2022] PUT /identity/v3/projects/83b467d008eb42c1b06fd8964ce90adc/users/9659619c4be34dd799f784405f31e062/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:23:17.459899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279f18f-75d9-4f97-8039-4012bf1e172f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:17.501105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6c7f9-6be5-4dc0-9381-e2fc1ab2ef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:17.501638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6c7f9-6be5-4dc0-9381-e2fc1ab2ef3e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:17.501884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6c7f9-6be5-4dc0-9381-e2fc1ab2ef3e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:17.502296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6c7f9-6be5-4dc0-9381-e2fc1ab2ef3e None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:17.502992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6c7f9-6be5-4dc0-9381-e2fc1ab2ef3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:17.503291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6c7f9-6be5-4dc0-9381-e2fc1ab2ef3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:17.504681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6c7f9-6be5-4dc0-9381-e2fc1ab2ef3e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:17.527224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 533/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:18.728354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471d1b4-739b-4223-ab34-ff7878a55d73 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:18.728934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471d1b4-739b-4223-ab34-ff7878a55d73 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:18.728934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471d1b4-739b-4223-ab34-ff7878a55d73 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:18.730344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 552/1085] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:18.736402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa068eb-9cf6-424d-a17a-2b39c9659bad None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:18.736807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa068eb-9cf6-424d-a17a-2b39c9659bad None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:18.736951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa068eb-9cf6-424d-a17a-2b39c9659bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:18.756323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0fa068eb-9cf6-424d-a17a-2b39c9659bad None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:18.757503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0fa068eb-9cf6-424d-a17a-2b39c9659bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:18.827429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 534/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:19.085326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471d1b4-739b-4223-ab34-ff7878a55d73 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:19.193160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471d1b4-739b-4223-ab34-ff7878a55d73 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:19.227790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cb9b0-0a59-46a6-b205-4c64876ec2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:19.228414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cb9b0-0a59-46a6-b205-4c64876ec2bb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:19.228693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cb9b0-0a59-46a6-b205-4c64876ec2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:19.228960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cb9b0-0a59-46a6-b205-4c64876ec2bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:19.240350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cb9b0-0a59-46a6-b205-4c64876ec2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-026cb9b0-0a59-46a6-b205-4c64876ec2bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:19.245095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:19.246010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:19.246968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 553/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Aug 31 04:23:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:19.252609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa068eb-9cf6-424d-a17a-2b39c9659bad None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:19.283766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:19.352643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa068eb-9cf6-424d-a17a-2b39c9659bad None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:19.364978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f633c-12df-4b0c-bfc4-9d0c06e04b1f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:19.365266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f633c-12df-4b0c-bfc4-9d0c06e04b1f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:19.365541 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f633c-12df-4b0c-bfc4-9d0c06e04b1f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:19.366894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 554/1088] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:19.372080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508044ca-b97b-42a1-8b69-28477f306dfa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:19.372240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508044ca-b97b-42a1-8b69-28477f306dfa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:19.372520 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508044ca-b97b-42a1-8b69-28477f306dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:19.388309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:19.388923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:19.389200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:19.389469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:19.390137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:19.390415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-508044ca-b97b-42a1-8b69-28477f306dfa None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:19.390523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:19.391625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-508044ca-b97b-42a1-8b69-28477f306dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:19.391826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:19.400030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 535/1089] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 31 04:23:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:19.406196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:19.421752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:19.463249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 555/1090] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 04:23:19.503789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:19.510993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f883c8-9dec-4a62-8b61-7e7c7ad064f3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:19.545556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:19.546181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:19.546451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:19.546696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:19.557481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:19.558947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:19.573830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:19.574832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:19.575725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:19.575725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:19.575725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:19.575725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 536/1091] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:19.581169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508044ca-b97b-42a1-8b69-28477f306dfa None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:19.683890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508044ca-b97b-42a1-8b69-28477f306dfa None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:19.718813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:19.719418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:19.719695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:19.719964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:19.720602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:19.720905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:19.722274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:19.726899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:19.734495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 556/1092] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:19.740674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:19.770022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:19.833019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f25e-c95e-4c40-bd99-43db6ac112dc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:19.842804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748810e1-e7d7-4c17-8764-e07a6b12b0c4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:19.867950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1bdb4-35b9-45a7-b8d6-c005467702cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:19.868575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1bdb4-35b9-45a7-b8d6-c005467702cb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:19.868851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1bdb4-35b9-45a7-b8d6-c005467702cb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:19.869118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1bdb4-35b9-45a7-b8d6-c005467702cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:19.877064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:19.877687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:19.877963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:19.878230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:19.879951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1bdb4-35b9-45a7-b8d6-c005467702cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a4e1bdb4-35b9-45a7-b8d6-c005467702cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:19.884736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:19.885903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:19.886986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 537/1093] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:19.887925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:19.889329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:19.892182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1bdb4-35b9-45a7-b8d6-c005467702cb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:23:19.900694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:19.901602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 04:23:19.902647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:19.904100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:19.904203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 557/1094] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Aug 31 04:23:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:19.907772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:19.997075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1bdb4-35b9-45a7-b8d6-c005467702cb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:19.997673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:20.010534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb80a4a-da6d-4877-ae83-00598806a22f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.030573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.031231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.031473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.031740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.032404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:20.032698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.034086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.042434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 538/1095] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:20.045375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.046057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.046395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.046753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.047555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:20.047857 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.048336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:20.049277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.067765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 558/1096] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Aug 31 04:23:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:20.074479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:20.114439 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:20.148006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3a415-ead9-4a8e-9325-55b6a90671d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.173625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec053d-94b6-4fdc-a3d9-1fa6d7d15cbb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.180519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea4b6b-05ff-4fb9-a16b-7604d45bc375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.181151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea4b6b-05ff-4fb9-a16b-7604d45bc375 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.181431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea4b6b-05ff-4fb9-a16b-7604d45bc375 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.181708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea4b6b-05ff-4fb9-a16b-7604d45bc375 None admin] PATH_INFO: `/v3/users/9659619c4be34dd799f784405f31e062` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.182995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea4b6b-05ff-4fb9-a16b-7604d45bc375 None admin] RBAC: Authorizing `identity:get_user(user_id=9659619c4be34dd799f784405f31e062)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.184454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea4b6b-05ff-4fb9-a16b-7604d45bc375 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.186212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 539/1097] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/users/9659619c4be34dd799f784405f31e062 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:20.192244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea4b6b-05ff-4fb9-a16b-7604d45bc375 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:20.209022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542fa48-0dca-4232-b7b6-65f49d5a3812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.209606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542fa48-0dca-4232-b7b6-65f49d5a3812 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.209872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542fa48-0dca-4232-b7b6-65f49d5a3812 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.210135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542fa48-0dca-4232-b7b6-65f49d5a3812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.225017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:20.275840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542fa48-0dca-4232-b7b6-65f49d5a3812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.277209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542fa48-0dca-4232-b7b6-65f49d5a3812 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.292920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea4b6b-05ff-4fb9-a16b-7604d45bc375 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.325459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08b94-c260-4b0d-a48f-09dc8cc37c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.326223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08b94-c260-4b0d-a48f-09dc8cc37c9f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.326496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08b94-c260-4b0d-a48f-09dc8cc37c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.326793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08b94-c260-4b0d-a48f-09dc8cc37c9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.330111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:20.373113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 559/1098] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:20.379037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542fa48-0dca-4232-b7b6-65f49d5a3812 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:20.395453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a08b94-c260-4b0d-a48f-09dc8cc37c9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.396924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a08b94-c260-4b0d-a48f-09dc8cc37c9f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.427533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:20.484539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542fa48-0dca-4232-b7b6-65f49d5a3812 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.505239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 540/1099] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:20.511313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08b94-c260-4b0d-a48f-09dc8cc37c9f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:20.524419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.525064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.525342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.525615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.536337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.537724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.544466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:20.552737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:20.553735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:23:20.554650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:20.554650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:23:20.554650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:20.554650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 560/1100] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:23:20.559612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:20.612187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08b94-c260-4b0d-a48f-09dc8cc37c9f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.646125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793b80d-9289-43e8-a73b-478b0ef9b22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.646780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b80d-9289-43e8-a73b-478b0ef9b22c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.647061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b80d-9289-43e8-a73b-478b0ef9b22c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.647325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793b80d-9289-43e8-a73b-478b0ef9b22c None admin] PATH_INFO: `/v3/projects/83b467d008eb42c1b06fd8964ce90adc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.648452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793b80d-9289-43e8-a73b-478b0ef9b22c None admin] RBAC: Authorizing `identity:get_project(project_id=83b467d008eb42c1b06fd8964ce90adc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.649856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793b80d-9289-43e8-a73b-478b0ef9b22c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.651545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 541/1101] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/projects/83b467d008eb42c1b06fd8964ce90adc => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:20.656420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:20.658239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793b80d-9289-43e8-a73b-478b0ef9b22c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:20.660465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e87bb-876c-435b-b4e5-e8f35f4be9eb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.696527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.697136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.697417 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.697683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.698322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:20.698621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.700076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.712206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 561/1102] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:20.719012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:20.758872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793b80d-9289-43e8-a73b-478b0ef9b22c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.761226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:20.795058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.795652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.795931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.796167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.796943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=83b467d008eb42c1b06fd8964ce90adc, user.id=9659619c4be34dd799f784405f31e062) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:20.797248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.798701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.805686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 542/1103] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/role_assignments?user.id=9659619c4be34dd799f784405f31e062&scope.project.id=83b467d008eb42c1b06fd8964ce90adc&role.id=241e99d365564c0fa95cb712f4004785 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:20.824038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9910be-6d49-4560-9780-5ccbbd63860e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:20.863608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:20.864613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:20.864890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:20.865166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:20.873647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:20.875195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:23:20.889794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:20.890949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:20.891854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 562/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:20.897279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:21.004909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369d18a-c390-445b-84d0-ea607a95c264 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:21.038461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737af71-e3d2-405e-a705-76baa0a28936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:21.039103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737af71-e3d2-405e-a705-76baa0a28936 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:21.039381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737af71-e3d2-405e-a705-76baa0a28936 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:21.039651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737af71-e3d2-405e-a705-76baa0a28936 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:21.040343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737af71-e3d2-405e-a705-76baa0a28936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:21.040641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737af71-e3d2-405e-a705-76baa0a28936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:21.042082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737af71-e3d2-405e-a705-76baa0a28936 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:21.054188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 543/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:23:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:21.060936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:21.082117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:21.169201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a6613-7a80-4128-9dcb-0cb1195306c8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:21.206122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62bd7c-d1c7-45cb-9253-42669f46596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:21.206742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62bd7c-d1c7-45cb-9253-42669f46596e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:21.207019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62bd7c-d1c7-45cb-9253-42669f46596e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:21.207286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62bd7c-d1c7-45cb-9253-42669f46596e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:21.208009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62bd7c-d1c7-45cb-9253-42669f46596e None admin] RBAC: Adding query filter params (role.id=d15c78e6c27741619010a21e9a172da7, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=708ce6cfc5f64d26a418b4339fd883fb) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:21.208308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62bd7c-d1c7-45cb-9253-42669f46596e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:21.209765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62bd7c-d1c7-45cb-9253-42669f46596e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:21.215695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 563/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:21 2022] GET /identity/v3/role_assignments?user.id=708ce6cfc5f64d26a418b4339fd883fb&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=d15c78e6c27741619010a21e9a172da7 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:22.709626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054b1af-ee6a-49ab-82c8-738ed204d3bc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:22.710471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054b1af-ee6a-49ab-82c8-738ed204d3bc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:22.710471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054b1af-ee6a-49ab-82c8-738ed204d3bc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:22.712678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 544/1107] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:22.720173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50615e-e77e-4d56-a2e5-8c77a1867e13 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:22.720585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50615e-e77e-4d56-a2e5-8c77a1867e13 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:22.720771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50615e-e77e-4d56-a2e5-8c77a1867e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:22.749481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4c50615e-e77e-4d56-a2e5-8c77a1867e13 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:22.750648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4c50615e-e77e-4d56-a2e5-8c77a1867e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:22.820344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 564/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:23.081171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054b1af-ee6a-49ab-82c8-738ed204d3bc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:23.114977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:23.199630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054b1af-ee6a-49ab-82c8-738ed204d3bc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:23.234532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38c4da-0304-4f48-a88f-7d18997be11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:23.235178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c4da-0304-4f48-a88f-7d18997be11c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:23.235452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c4da-0304-4f48-a88f-7d18997be11c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:23.235721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c4da-0304-4f48-a88f-7d18997be11c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:23.236952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38c4da-0304-4f48-a88f-7d18997be11c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:23.238389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38c4da-0304-4f48-a88f-7d18997be11c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:23.240055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 545/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:23.248777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50615e-e77e-4d56-a2e5-8c77a1867e13 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:23.264636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:23.356981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50615e-e77e-4d56-a2e5-8c77a1867e13 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:23.393292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaef544-cc71-4a11-9ab0-f921b89b940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:23.393900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaef544-cc71-4a11-9ab0-f921b89b940f None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:23.394173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaef544-cc71-4a11-9ab0-f921b89b940f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:23.394444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaef544-cc71-4a11-9ab0-f921b89b940f None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:23.395117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaef544-cc71-4a11-9ab0-f921b89b940f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:23.396554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaef544-cc71-4a11-9ab0-f921b89b940f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:23.424731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 565/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:23:23 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:23:25.333103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee9a78-e561-4507-b2c9-9abf384136bd None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:25.333909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee9a78-e561-4507-b2c9-9abf384136bd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:25.333909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee9a78-e561-4507-b2c9-9abf384136bd None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:25.336090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 546/1111] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:25.343833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e23011-81ab-4e49-a2a1-43f4481d521c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:25.344526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e23011-81ab-4e49-a2a1-43f4481d521c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:25.344734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e23011-81ab-4e49-a2a1-43f4481d521c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:25.369517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b1e23011-81ab-4e49-a2a1-43f4481d521c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:25.370740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b1e23011-81ab-4e49-a2a1-43f4481d521c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:25.445100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:25.453119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 566/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:25.745569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee9a78-e561-4507-b2c9-9abf384136bd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:25.856275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:25.863678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee9a78-e561-4507-b2c9-9abf384136bd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:25.900580 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c55cec-361f-4c6d-86da-696668ca12c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:25.901230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c55cec-361f-4c6d-86da-696668ca12c5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:25.901506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c55cec-361f-4c6d-86da-696668ca12c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:25.901775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c55cec-361f-4c6d-86da-696668ca12c5 None admin] PATH_INFO: `/v3/roles/d06115dbab78488182bcb432bf9bec11` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:25.903555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c55cec-361f-4c6d-86da-696668ca12c5 None admin] RBAC: Authorizing `identity:get_role(role_id=d06115dbab78488182bcb432bf9bec11)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:25.905320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c55cec-361f-4c6d-86da-696668ca12c5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:25.906686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 547/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:25 2022] GET /identity/v3/roles/d06115dbab78488182bcb432bf9bec11 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:25.913323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e23011-81ab-4e49-a2a1-43f4481d521c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:26.017550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e23011-81ab-4e49-a2a1-43f4481d521c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:26.052013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656c023-7c47-4cfd-990c-ab08fffde66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:26.052639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656c023-7c47-4cfd-990c-ab08fffde66d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:26.052911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656c023-7c47-4cfd-990c-ab08fffde66d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:26.053176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656c023-7c47-4cfd-990c-ab08fffde66d None admin] PATH_INFO: `/v3/users/dac5b7aaa7cf44938467d79417cbd05f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:26.069165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656c023-7c47-4cfd-990c-ab08fffde66d None admin] RBAC: Authorizing `identity:get_user(user_id=dac5b7aaa7cf44938467d79417cbd05f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:26.071015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656c023-7c47-4cfd-990c-ab08fffde66d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:26.072659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 567/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:25 2022] GET /identity/v3/users/dac5b7aaa7cf44938467d79417cbd05f => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:26.079404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c55cec-361f-4c6d-86da-696668ca12c5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:26.206353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c55cec-361f-4c6d-86da-696668ca12c5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:26.245640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bf40b-ecc6-4094-8cf8-5e036a452cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:26.246374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bf40b-ecc6-4094-8cf8-5e036a452cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:26.246653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bf40b-ecc6-4094-8cf8-5e036a452cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:26.246954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bf40b-ecc6-4094-8cf8-5e036a452cd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:26.289309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:26.335024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bf40b-ecc6-4094-8cf8-5e036a452cd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:26.336477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bf40b-ecc6-4094-8cf8-5e036a452cd4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:26.450470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 548/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:26.459081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656c023-7c47-4cfd-990c-ab08fffde66d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:26.521791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:26.576672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656c023-7c47-4cfd-990c-ab08fffde66d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:26.615553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65749f-06db-4bfa-95ac-cdf84a3d03be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:26.616187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65749f-06db-4bfa-95ac-cdf84a3d03be None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:26.616464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65749f-06db-4bfa-95ac-cdf84a3d03be None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:26.616737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65749f-06db-4bfa-95ac-cdf84a3d03be None admin] PATH_INFO: `/v3/projects/83b467d008eb42c1b06fd8964ce90adc` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:26.617978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65749f-06db-4bfa-95ac-cdf84a3d03be None admin] RBAC: Authorizing `identity:get_project(project_id=83b467d008eb42c1b06fd8964ce90adc)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:26.619456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65749f-06db-4bfa-95ac-cdf84a3d03be None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:26.621102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 568/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:26 2022] GET /identity/v3/projects/83b467d008eb42c1b06fd8964ce90adc => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:26.628536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bf40b-ecc6-4094-8cf8-5e036a452cd4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:26.700892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:26.744648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bf40b-ecc6-4094-8cf8-5e036a452cd4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:26.789698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19d504-7f0a-428d-a0f6-ab3969662a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:26.790632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19d504-7f0a-428d-a0f6-ab3969662a98 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:26.791135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19d504-7f0a-428d-a0f6-ab3969662a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:26.791573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19d504-7f0a-428d-a0f6-ab3969662a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:26.792748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19d504-7f0a-428d-a0f6-ab3969662a98 None admin] RBAC: Adding query filter params (role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=83b467d008eb42c1b06fd8964ce90adc, user.id=dac5b7aaa7cf44938467d79417cbd05f) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:26.793273 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19d504-7f0a-428d-a0f6-ab3969662a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:26.795864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19d504-7f0a-428d-a0f6-ab3969662a98 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:26.801366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 549/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:26 2022] GET /identity/v3/role_assignments?user.id=dac5b7aaa7cf44938467d79417cbd05f&scope.project.id=83b467d008eb42c1b06fd8964ce90adc&role.id=d06115dbab78488182bcb432bf9bec11 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:28.640859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b846a4-febe-458a-8be2-bf1b64441587 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:28.641426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b846a4-febe-458a-8be2-bf1b64441587 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:28.641426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b846a4-febe-458a-8be2-bf1b64441587 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:28.643543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 569/1118] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:28.652416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cab801-effd-44a6-b70a-e9dcf0d3a540 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:28.652824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cab801-effd-44a6-b70a-e9dcf0d3a540 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:28.653005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cab801-effd-44a6-b70a-e9dcf0d3a540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:28.674420 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-81cab801-effd-44a6-b70a-e9dcf0d3a540 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:28.675626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-81cab801-effd-44a6-b70a-e9dcf0d3a540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:28.756454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 550/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:29.019586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b846a4-febe-458a-8be2-bf1b64441587 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:29.133476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b846a4-febe-458a-8be2-bf1b64441587 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:29.157322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:29.174832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b5aec-2ad9-4e20-9b8c-58037127243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:29.175471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b5aec-2ad9-4e20-9b8c-58037127243e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:29.175743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b5aec-2ad9-4e20-9b8c-58037127243e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:29.175999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b5aec-2ad9-4e20-9b8c-58037127243e None admin] PATH_INFO: `/v3/roles/d06115dbab78488182bcb432bf9bec11` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:29.177486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b5aec-2ad9-4e20-9b8c-58037127243e None admin] RBAC: Authorizing `identity:get_role(role_id=d06115dbab78488182bcb432bf9bec11)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:29.178900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b5aec-2ad9-4e20-9b8c-58037127243e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:29.180260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 570/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:29 2022] GET /identity/v3/roles/d06115dbab78488182bcb432bf9bec11 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:29.186259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cab801-effd-44a6-b70a-e9dcf0d3a540 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:29.245105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:29.289163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cab801-effd-44a6-b70a-e9dcf0d3a540 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:29.324407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d092c19-f7f5-4bdc-a0e8-cdd3e081ced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:29.325025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d092c19-f7f5-4bdc-a0e8-cdd3e081ced5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:29.325304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d092c19-f7f5-4bdc-a0e8-cdd3e081ced5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:29.325570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d092c19-f7f5-4bdc-a0e8-cdd3e081ced5 None admin] PATH_INFO: `/v3/users/dac5b7aaa7cf44938467d79417cbd05f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:29.326741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d092c19-f7f5-4bdc-a0e8-cdd3e081ced5 None admin] RBAC: Authorizing `identity:get_user(user_id=dac5b7aaa7cf44938467d79417cbd05f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:29.328247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d092c19-f7f5-4bdc-a0e8-cdd3e081ced5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:29.329822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 551/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:29 2022] GET /identity/v3/users/dac5b7aaa7cf44938467d79417cbd05f => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:29.335811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b5aec-2ad9-4e20-9b8c-58037127243e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:29.440639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b5aec-2ad9-4e20-9b8c-58037127243e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:29.482593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7da499-f871-48a9-a2c2-c29bb38b0c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:29.483460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da499-f871-48a9-a2c2-c29bb38b0c6e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:29.483740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da499-f871-48a9-a2c2-c29bb38b0c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:29.484014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da499-f871-48a9-a2c2-c29bb38b0c6e None admin] PATH_INFO: `/v3/projects/83b467d008eb42c1b06fd8964ce90adc` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:29.485244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7da499-f871-48a9-a2c2-c29bb38b0c6e None admin] RBAC: Authorizing `identity:get_project(project_id=83b467d008eb42c1b06fd8964ce90adc)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:29.486679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7da499-f871-48a9-a2c2-c29bb38b0c6e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:29.488326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 571/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:29 2022] GET /identity/v3/projects/83b467d008eb42c1b06fd8964ce90adc => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:29.494298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d092c19-f7f5-4bdc-a0e8-cdd3e081ced5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:29.612682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d092c19-f7f5-4bdc-a0e8-cdd3e081ced5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:29.639280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:29.653078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fad547-ec5b-40fd-b5dd-05464eae7261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:29.653682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad547-ec5b-40fd-b5dd-05464eae7261 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:29.653994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad547-ec5b-40fd-b5dd-05464eae7261 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:29.654262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad547-ec5b-40fd-b5dd-05464eae7261 None admin] PATH_INFO: `/v3/projects/83b467d008eb42c1b06fd8964ce90adc/users/dac5b7aaa7cf44938467d79417cbd05f/roles/d06115dbab78488182bcb432bf9bec11` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:29.659244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fad547-ec5b-40fd-b5dd-05464eae7261 None admin] RBAC: Authorizing `identity:create_grant(project_id=83b467d008eb42c1b06fd8964ce90adc, user_id=dac5b7aaa7cf44938467d79417cbd05f, role_id=d06115dbab78488182bcb432bf9bec11)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:29.660700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fad547-ec5b-40fd-b5dd-05464eae7261 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:29.677278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 552/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:23:29 2022] PUT /identity/v3/projects/83b467d008eb42c1b06fd8964ce90adc/users/dac5b7aaa7cf44938467d79417cbd05f/roles/d06115dbab78488182bcb432bf9bec11 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:23:31.528879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec0861-6b9e-4ddb-8610-75b54c2a9d5d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:31.529635 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec0861-6b9e-4ddb-8610-75b54c2a9d5d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:31.530316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec0861-6b9e-4ddb-8610-75b54c2a9d5d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:31.532064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 572/1124] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:31.537398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d738e-d65c-49c6-a621-c1c557aa651d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:31.537822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d738e-d65c-49c6-a621-c1c557aa651d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:31.537919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d738e-d65c-49c6-a621-c1c557aa651d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:31.556833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ce1d738e-d65c-49c6-a621-c1c557aa651d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:31.557989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce1d738e-d65c-49c6-a621-c1c557aa651d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:31.627262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 553/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:31.890155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec0861-6b9e-4ddb-8610-75b54c2a9d5d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:31.968134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:32.018997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec0861-6b9e-4ddb-8610-75b54c2a9d5d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:32.054822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecda37f-183d-4abf-99ef-3f5f4f797e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:32.055447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecda37f-183d-4abf-99ef-3f5f4f797e41 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:32.055731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecda37f-183d-4abf-99ef-3f5f4f797e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:32.056010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecda37f-183d-4abf-99ef-3f5f4f797e41 None admin] PATH_INFO: `/v3/roles/d06115dbab78488182bcb432bf9bec11` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:32.057536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecda37f-183d-4abf-99ef-3f5f4f797e41 None admin] RBAC: Authorizing `identity:get_role(role_id=d06115dbab78488182bcb432bf9bec11)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:32.058960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecda37f-183d-4abf-99ef-3f5f4f797e41 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:32.060469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 573/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:31 2022] GET /identity/v3/roles/d06115dbab78488182bcb432bf9bec11 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:32.067486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d738e-d65c-49c6-a621-c1c557aa651d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:32.168564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d738e-d65c-49c6-a621-c1c557aa651d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:32.185137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:32.204293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b056b-9584-4382-bd69-35b64d9a9e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:32.204919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b056b-9584-4382-bd69-35b64d9a9e42 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:32.205194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b056b-9584-4382-bd69-35b64d9a9e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:32.205464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b056b-9584-4382-bd69-35b64d9a9e42 None admin] PATH_INFO: `/v3/users/dac5b7aaa7cf44938467d79417cbd05f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:32.206598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b056b-9584-4382-bd69-35b64d9a9e42 None admin] RBAC: Authorizing `identity:get_user(user_id=dac5b7aaa7cf44938467d79417cbd05f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:32.208073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b056b-9584-4382-bd69-35b64d9a9e42 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:32.209674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 554/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:32 2022] GET /identity/v3/users/dac5b7aaa7cf44938467d79417cbd05f => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:32.219303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecda37f-183d-4abf-99ef-3f5f4f797e41 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:32.323772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecda37f-183d-4abf-99ef-3f5f4f797e41 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:32.351024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:32.365284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c76e29-1301-4e66-86e5-a5403df02a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:32.366201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76e29-1301-4e66-86e5-a5403df02a48 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:32.366620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76e29-1301-4e66-86e5-a5403df02a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:32.367092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76e29-1301-4e66-86e5-a5403df02a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:32.440758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c76e29-1301-4e66-86e5-a5403df02a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:32.442147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c76e29-1301-4e66-86e5-a5403df02a48 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:32.549449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 574/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:32.558592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b056b-9584-4382-bd69-35b64d9a9e42 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:32.662502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b056b-9584-4382-bd69-35b64d9a9e42 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:32.698414 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87882fa-13ec-4bcd-9d79-18ccccd67a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:32.698988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87882fa-13ec-4bcd-9d79-18ccccd67a8e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:32.699258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87882fa-13ec-4bcd-9d79-18ccccd67a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:32.699524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87882fa-13ec-4bcd-9d79-18ccccd67a8e None admin] PATH_INFO: `/v3/projects/83b467d008eb42c1b06fd8964ce90adc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:32.700674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87882fa-13ec-4bcd-9d79-18ccccd67a8e None admin] RBAC: Authorizing `identity:get_project(project_id=83b467d008eb42c1b06fd8964ce90adc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:32.702178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87882fa-13ec-4bcd-9d79-18ccccd67a8e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:32.703814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 555/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:32 2022] GET /identity/v3/projects/83b467d008eb42c1b06fd8964ce90adc => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:32.713244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c76e29-1301-4e66-86e5-a5403df02a48 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:32.743205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:32.816401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c76e29-1301-4e66-86e5-a5403df02a48 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:32.851942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da84e65-2b6c-4fa4-86d5-dfc974b628bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:32.852538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da84e65-2b6c-4fa4-86d5-dfc974b628bc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:32.852813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da84e65-2b6c-4fa4-86d5-dfc974b628bc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:32.853082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da84e65-2b6c-4fa4-86d5-dfc974b628bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:32.853803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da84e65-2b6c-4fa4-86d5-dfc974b628bc None admin] RBAC: Adding query filter params (role.id=d06115dbab78488182bcb432bf9bec11, scope.project.id=83b467d008eb42c1b06fd8964ce90adc, user.id=dac5b7aaa7cf44938467d79417cbd05f) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:32.854095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da84e65-2b6c-4fa4-86d5-dfc974b628bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:32.855546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da84e65-2b6c-4fa4-86d5-dfc974b628bc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:32.861827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 575/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:32 2022] GET /identity/v3/role_assignments?user.id=dac5b7aaa7cf44938467d79417cbd05f&scope.project.id=83b467d008eb42c1b06fd8964ce90adc&role.id=d06115dbab78488182bcb432bf9bec11 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:34.753130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139073ab-5a12-466a-8fc1-ec1fc94cf941 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:34.753690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139073ab-5a12-466a-8fc1-ec1fc94cf941 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:34.753690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139073ab-5a12-466a-8fc1-ec1fc94cf941 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:34.755173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 556/1131] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:34.763390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a12bc-d301-45b2-8bba-25fec4f62da8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:34.763839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a12bc-d301-45b2-8bba-25fec4f62da8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:34.763939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a12bc-d301-45b2-8bba-25fec4f62da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:34.787966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6c4a12bc-d301-45b2-8bba-25fec4f62da8 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:34.789151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6c4a12bc-d301-45b2-8bba-25fec4f62da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:34.871852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 576/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:35.152572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139073ab-5a12-466a-8fc1-ec1fc94cf941 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:35.210616 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:35.264338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139073ab-5a12-466a-8fc1-ec1fc94cf941 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:35.299604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbdafe-20e0-4a03-8b21-0a5af1d0b64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:35.300212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbdafe-20e0-4a03-8b21-0a5af1d0b64f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:35.300484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbdafe-20e0-4a03-8b21-0a5af1d0b64f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:35.300757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbdafe-20e0-4a03-8b21-0a5af1d0b64f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:35.301908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbdafe-20e0-4a03-8b21-0a5af1d0b64f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:35.303376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbdafe-20e0-4a03-8b21-0a5af1d0b64f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:35.304902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 557/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:35.311260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a12bc-d301-45b2-8bba-25fec4f62da8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:35.338945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:35.423666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a12bc-d301-45b2-8bba-25fec4f62da8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:35.471873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc170-cdf9-4599-a678-4ff0433a60a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:35.472495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc170-cdf9-4599-a678-4ff0433a60a0 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:35.472761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc170-cdf9-4599-a678-4ff0433a60a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:35.473025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc170-cdf9-4599-a678-4ff0433a60a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:35.473660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc170-cdf9-4599-a678-4ff0433a60a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:35.475133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc170-cdf9-4599-a678-4ff0433a60a0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:35.507452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 577/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:23:35 2022] POST /identity/v3/projects => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:23:37.405269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31e55-a8c4-47ef-b0cd-3f8a6ecc6782 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:37.406094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31e55-a8c4-47ef-b0cd-3f8a6ecc6782 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:37.406094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31e55-a8c4-47ef-b0cd-3f8a6ecc6782 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:37.408380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 558/1135] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:37.414892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cdeae-add4-4d82-b533-f1e3d36b362d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:37.415318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cdeae-add4-4d82-b533-f1e3d36b362d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:37.415475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cdeae-add4-4d82-b533-f1e3d36b362d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:37.445423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3d5cdeae-add4-4d82-b533-f1e3d36b362d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:37.447381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3d5cdeae-add4-4d82-b533-f1e3d36b362d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:37.531122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 578/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:37.794081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31e55-a8c4-47ef-b0cd-3f8a6ecc6782 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:37.903967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31e55-a8c4-47ef-b0cd-3f8a6ecc6782 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:37.923860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:37.939406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffa1c6-403d-4a06-bdcd-5120e9063ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:37.940021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffa1c6-403d-4a06-bdcd-5120e9063ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:37.940296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffa1c6-403d-4a06-bdcd-5120e9063ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:37.940560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffa1c6-403d-4a06-bdcd-5120e9063ee9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:37.941759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffa1c6-403d-4a06-bdcd-5120e9063ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:37.943252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffa1c6-403d-4a06-bdcd-5120e9063ee9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:37.945155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 559/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:37.951700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cdeae-add4-4d82-b533-f1e3d36b362d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:37.955839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:38.056822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cdeae-add4-4d82-b533-f1e3d36b362d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:38.098614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd32e55-0e78-4a73-aaa5-e7ac2498ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:38.099254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd32e55-0e78-4a73-aaa5-e7ac2498ac9b None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:38.099530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd32e55-0e78-4a73-aaa5-e7ac2498ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:38.099803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd32e55-0e78-4a73-aaa5-e7ac2498ac9b None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:38.100447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd32e55-0e78-4a73-aaa5-e7ac2498ac9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:38.101887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd32e55-0e78-4a73-aaa5-e7ac2498ac9b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:38.127060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 579/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:23:37 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:23:40.048634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9932a28-c9ab-4ddc-93f7-fbc02fd22571 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:40.049530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9932a28-c9ab-4ddc-93f7-fbc02fd22571 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:40.049530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9932a28-c9ab-4ddc-93f7-fbc02fd22571 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:40.051669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 560/1139] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:40.060409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0cd9d-a674-4f27-bb68-5fa6262cb12c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:40.061116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0cd9d-a674-4f27-bb68-5fa6262cb12c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:40.061533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0cd9d-a674-4f27-bb68-5fa6262cb12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:40.088316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6db0cd9d-a674-4f27-bb68-5fa6262cb12c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:40.089505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6db0cd9d-a674-4f27-bb68-5fa6262cb12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:40.165900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:40.181922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 580/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:40.441531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9932a28-c9ab-4ddc-93f7-fbc02fd22571 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:40.555556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9932a28-c9ab-4ddc-93f7-fbc02fd22571 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:40.592375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3f02d-9b3b-4baa-8fc1-913a1142f0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:40.592997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f02d-9b3b-4baa-8fc1-913a1142f0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:40.593275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f02d-9b3b-4baa-8fc1-913a1142f0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:40.593548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f02d-9b3b-4baa-8fc1-913a1142f0d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:40.604424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3f02d-9b3b-4baa-8fc1-913a1142f0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2cd3f02d-9b3b-4baa-8fc1-913a1142f0d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:40.609217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:40.610349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:40.611326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 561/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:23:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:40.617119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0cd9d-a674-4f27-bb68-5fa6262cb12c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:40.724046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0cd9d-a674-4f27-bb68-5fa6262cb12c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:40.758679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:40.759307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:40.759572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:40.759837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:40.760491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:40.760784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:40.762129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:40.770215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 581/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:23:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:40.776624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3f02d-9b3b-4baa-8fc1-913a1142f0d0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:40.837863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:40.893157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3f02d-9b3b-4baa-8fc1-913a1142f0d0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:40.936201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712917d-7b9e-436a-ab0b-5c2dddf4cbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:40.936859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712917d-7b9e-436a-ab0b-5c2dddf4cbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:40.937135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712917d-7b9e-436a-ab0b-5c2dddf4cbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:40.937402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712917d-7b9e-436a-ab0b-5c2dddf4cbe8 None admin] PATH_INFO: `/v3/users/903ccb0042194070a5bc0fd0b17da1a8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:40.951074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712917d-7b9e-436a-ab0b-5c2dddf4cbe8 None admin] RBAC: Authorizing `identity:get_user(user_id=903ccb0042194070a5bc0fd0b17da1a8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:40.952512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712917d-7b9e-436a-ab0b-5c2dddf4cbe8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:40.954189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 562/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:40 2022] GET /identity/v3/users/903ccb0042194070a5bc0fd0b17da1a8 => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:40.960449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:41.014271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:41.062911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c644288-e2cb-4fc3-8889-c6bc4ed97cd9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:41.100881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1dbd0-5d79-4993-a2e5-946d5622be3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:41.101490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dbd0-5d79-4993-a2e5-946d5622be3a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:41.101767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dbd0-5d79-4993-a2e5-946d5622be3a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:41.102035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dbd0-5d79-4993-a2e5-946d5622be3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:41.175792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1dbd0-5d79-4993-a2e5-946d5622be3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:41.177236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1dbd0-5d79-4993-a2e5-946d5622be3a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:41.228920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:41.286638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 582/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:41.295502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712917d-7b9e-436a-ab0b-5c2dddf4cbe8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:41.414043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712917d-7b9e-436a-ab0b-5c2dddf4cbe8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:41.421242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:41.459392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2bcb2-bba7-4b11-8f5e-d6355693d98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:41.460024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2bcb2-bba7-4b11-8f5e-d6355693d98d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:41.460303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2bcb2-bba7-4b11-8f5e-d6355693d98d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:41.460574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2bcb2-bba7-4b11-8f5e-d6355693d98d None admin] PATH_INFO: `/v3/projects/65b872d2076749a5810ed7e93bb722e4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:41.461821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2bcb2-bba7-4b11-8f5e-d6355693d98d None admin] RBAC: Authorizing `identity:get_project(project_id=65b872d2076749a5810ed7e93bb722e4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:41.463449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2bcb2-bba7-4b11-8f5e-d6355693d98d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:41.465080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 563/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:41 2022] GET /identity/v3/projects/65b872d2076749a5810ed7e93bb722e4 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:23:41.472626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1dbd0-5d79-4993-a2e5-946d5622be3a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:41.599237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1dbd0-5d79-4993-a2e5-946d5622be3a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:41.635733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e6695-dc57-437f-974e-7ec7d03ac623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:41.636333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e6695-dc57-437f-974e-7ec7d03ac623 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:41.636610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e6695-dc57-437f-974e-7ec7d03ac623 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:41.636891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e6695-dc57-437f-974e-7ec7d03ac623 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:41.637613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e6695-dc57-437f-974e-7ec7d03ac623 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=65b872d2076749a5810ed7e93bb722e4, user.id=903ccb0042194070a5bc0fd0b17da1a8) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:41.637921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e6695-dc57-437f-974e-7ec7d03ac623 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:41.639393 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e6695-dc57-437f-974e-7ec7d03ac623 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:41.645008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 583/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:41 2022] GET /identity/v3/role_assignments?user.id=903ccb0042194070a5bc0fd0b17da1a8&scope.project.id=65b872d2076749a5810ed7e93bb722e4&role.id=241e99d365564c0fa95cb712f4004785 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:43.510287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548da80-2f2f-494f-a37e-a50ea97c59dd None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:43.511138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548da80-2f2f-494f-a37e-a50ea97c59dd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:43.511138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548da80-2f2f-494f-a37e-a50ea97c59dd None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:43.513189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 564/1147] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:43.522026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a475f3a-cd4d-4562-9528-ee18aec9f87c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:43.522432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a475f3a-cd4d-4562-9528-ee18aec9f87c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:43.522632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a475f3a-cd4d-4562-9528-ee18aec9f87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:43.545338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6a475f3a-cd4d-4562-9528-ee18aec9f87c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:43.546529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6a475f3a-cd4d-4562-9528-ee18aec9f87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:43.557797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:43.615920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 584/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:43.879968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548da80-2f2f-494f-a37e-a50ea97c59dd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:44.007588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548da80-2f2f-494f-a37e-a50ea97c59dd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:44.044621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b70f7-a69c-40e6-b423-0647c446796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:44.045228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b70f7-a69c-40e6-b423-0647c446796a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:44.045497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b70f7-a69c-40e6-b423-0647c446796a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:44.045756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b70f7-a69c-40e6-b423-0647c446796a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:44.056500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b70f7-a69c-40e6-b423-0647c446796a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-678b70f7-a69c-40e6-b423-0647c446796a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:44.061250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:44.062295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:44.064127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 565/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:23:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:44.068629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a475f3a-cd4d-4562-9528-ee18aec9f87c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:44.173801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a475f3a-cd4d-4562-9528-ee18aec9f87c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:44.210875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:44.211476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:44.211747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:44.212008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:44.212661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:44.212949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:44.214330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:44.223355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 585/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:44.230039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b70f7-a69c-40e6-b423-0647c446796a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:44.275931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:44.351769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b70f7-a69c-40e6-b423-0647c446796a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:44.389217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9c58f-4324-40a7-b8b4-7b64ede26c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:44.389829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9c58f-4324-40a7-b8b4-7b64ede26c86 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:44.390102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9c58f-4324-40a7-b8b4-7b64ede26c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:44.390371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9c58f-4324-40a7-b8b4-7b64ede26c86 None admin] PATH_INFO: `/v3/users/903ccb0042194070a5bc0fd0b17da1a8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:44.391547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9c58f-4324-40a7-b8b4-7b64ede26c86 None admin] RBAC: Authorizing `identity:get_user(user_id=903ccb0042194070a5bc0fd0b17da1a8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:44.393008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9c58f-4324-40a7-b8b4-7b64ede26c86 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:44.394573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 566/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:44 2022] GET /identity/v3/users/903ccb0042194070a5bc0fd0b17da1a8 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:44.404117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:44.417260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:44.520638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb03861-986b-4e48-afef-9e103e8d2a14 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:44.555887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140721d-567b-4601-8ac9-a2a168b29c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:44.556473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140721d-567b-4601-8ac9-a2a168b29c5c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:44.556764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140721d-567b-4601-8ac9-a2a168b29c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:44.557030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140721d-567b-4601-8ac9-a2a168b29c5c None admin] PATH_INFO: `/v3/projects/65b872d2076749a5810ed7e93bb722e4` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:44.558172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140721d-567b-4601-8ac9-a2a168b29c5c None admin] RBAC: Authorizing `identity:get_project(project_id=65b872d2076749a5810ed7e93bb722e4)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:44.559634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140721d-567b-4601-8ac9-a2a168b29c5c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:44.561247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 586/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:44 2022] GET /identity/v3/projects/65b872d2076749a5810ed7e93bb722e4 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:44.571289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9c58f-4324-40a7-b8b4-7b64ede26c86 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:44.684389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9c58f-4324-40a7-b8b4-7b64ede26c86 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:44.686105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:44.722159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ac5ee-66d4-44fb-99ee-124731bc6a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:44.722833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ac5ee-66d4-44fb-99ee-124731bc6a5b None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:44.723107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ac5ee-66d4-44fb-99ee-124731bc6a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:44.723376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ac5ee-66d4-44fb-99ee-124731bc6a5b None admin] PATH_INFO: `/v3/projects/65b872d2076749a5810ed7e93bb722e4/users/903ccb0042194070a5bc0fd0b17da1a8/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:44.725322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ac5ee-66d4-44fb-99ee-124731bc6a5b None admin] RBAC: Authorizing `identity:create_grant(project_id=65b872d2076749a5810ed7e93bb722e4, user_id=903ccb0042194070a5bc0fd0b17da1a8, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:44.726749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ac5ee-66d4-44fb-99ee-124731bc6a5b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:44.745666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 567/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:23:44 2022] PUT /identity/v3/projects/65b872d2076749a5810ed7e93bb722e4/users/903ccb0042194070a5bc0fd0b17da1a8/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:23:46.590334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c232d-bf07-49a9-b36f-a0507176cb48 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:46.591066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c232d-bf07-49a9-b36f-a0507176cb48 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:46.591066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c232d-bf07-49a9-b36f-a0507176cb48 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:46.592496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 587/1154] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:46.599150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb5f5f-825d-479c-b8ce-458c33ba40d9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:46.599910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb5f5f-825d-479c-b8ce-458c33ba40d9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:46.600095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb5f5f-825d-479c-b8ce-458c33ba40d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:46.629671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-70bb5f5f-825d-479c-b8ce-458c33ba40d9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:46.630884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-70bb5f5f-825d-479c-b8ce-458c33ba40d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:46.700649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 568/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:46.974639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c232d-bf07-49a9-b36f-a0507176cb48 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:47.068081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:47.090146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c232d-bf07-49a9-b36f-a0507176cb48 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:47.125455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf5e10-c579-4a04-9126-cd1de625fe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:47.126078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf5e10-c579-4a04-9126-cd1de625fe95 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:47.126354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf5e10-c579-4a04-9126-cd1de625fe95 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:47.126628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf5e10-c579-4a04-9126-cd1de625fe95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:47.137942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf5e10-c579-4a04-9126-cd1de625fe95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-fccf5e10-c579-4a04-9126-cd1de625fe95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:23:47.142852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:47.143936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:23:47.144795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 588/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:23:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:47.150290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb5f5f-825d-479c-b8ce-458c33ba40d9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:47.248921 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:47.259804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb5f5f-825d-479c-b8ce-458c33ba40d9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:47.307230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:47.307831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:47.308108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:47.308374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:47.309044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:47.309349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:47.310751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:47.319164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 569/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:47.328070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf5e10-c579-4a04-9126-cd1de625fe95 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:47.447073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf5e10-c579-4a04-9126-cd1de625fe95 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:47.482587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc8c25-b347-4a4b-a5ac-3b871b1fdf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:47.483238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc8c25-b347-4a4b-a5ac-3b871b1fdf81 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:47.483515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc8c25-b347-4a4b-a5ac-3b871b1fdf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:47.483780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc8c25-b347-4a4b-a5ac-3b871b1fdf81 None admin] PATH_INFO: `/v3/users/903ccb0042194070a5bc0fd0b17da1a8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:47.485021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc8c25-b347-4a4b-a5ac-3b871b1fdf81 None admin] RBAC: Authorizing `identity:get_user(user_id=903ccb0042194070a5bc0fd0b17da1a8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:47.486462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc8c25-b347-4a4b-a5ac-3b871b1fdf81 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:47.488100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 589/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:23:47 2022] GET /identity/v3/users/903ccb0042194070a5bc0fd0b17da1a8 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:47.495147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:47.601600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774e17a-256e-4bc2-bc6b-2c5e690bb88f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:47.636628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62ee64-f67b-46a1-85c7-6788c8790343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:47.637219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ee64-f67b-46a1-85c7-6788c8790343 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:47.637489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ee64-f67b-46a1-85c7-6788c8790343 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:47.637762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ee64-f67b-46a1-85c7-6788c8790343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:47.663017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:47.709917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62ee64-f67b-46a1-85c7-6788c8790343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:47.711398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62ee64-f67b-46a1-85c7-6788c8790343 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:47.813605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 570/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:47.823886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc8c25-b347-4a4b-a5ac-3b871b1fdf81 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:47.830099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:47.942075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc8c25-b347-4a4b-a5ac-3b871b1fdf81 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:47.983528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a4c2b-de0e-4af0-a537-181303f43d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:47.984154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a4c2b-de0e-4af0-a537-181303f43d1c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:47.984429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a4c2b-de0e-4af0-a537-181303f43d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:47.984696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a4c2b-de0e-4af0-a537-181303f43d1c None admin] PATH_INFO: `/v3/projects/65b872d2076749a5810ed7e93bb722e4` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:47.985805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a4c2b-de0e-4af0-a537-181303f43d1c None admin] RBAC: Authorizing `identity:get_project(project_id=65b872d2076749a5810ed7e93bb722e4)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:47.987389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a4c2b-de0e-4af0-a537-181303f43d1c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:47.989034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 590/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:23:47 2022] GET /identity/v3/projects/65b872d2076749a5810ed7e93bb722e4 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:47.995921 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62ee64-f67b-46a1-85c7-6788c8790343 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:48.054728 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:48.103988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62ee64-f67b-46a1-85c7-6788c8790343 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:48.138309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1035d4-73d2-4f18-8806-16dda9cec4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:48.138932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1035d4-73d2-4f18-8806-16dda9cec4ba None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:48.139208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1035d4-73d2-4f18-8806-16dda9cec4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:48.139470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1035d4-73d2-4f18-8806-16dda9cec4ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:48.140192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1035d4-73d2-4f18-8806-16dda9cec4ba None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=65b872d2076749a5810ed7e93bb722e4, user.id=903ccb0042194070a5bc0fd0b17da1a8) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:48.140493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1035d4-73d2-4f18-8806-16dda9cec4ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:48.141916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1035d4-73d2-4f18-8806-16dda9cec4ba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:48.147855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 571/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:23:47 2022] GET /identity/v3/role_assignments?user.id=903ccb0042194070a5bc0fd0b17da1a8&scope.project.id=65b872d2076749a5810ed7e93bb722e4&role.id=241e99d365564c0fa95cb712f4004785 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:50.039628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa670167-3c54-472f-a987-639f41c877a4 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:50.040192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa670167-3c54-472f-a987-639f41c877a4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:50.040192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa670167-3c54-472f-a987-639f41c877a4 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:50.041512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 591/1162] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:50.047881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2892f-e332-4515-a39a-76831ab3b832 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:50.048586 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2892f-e332-4515-a39a-76831ab3b832 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:50.049022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2892f-e332-4515-a39a-76831ab3b832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:50.069416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-18b2892f-e332-4515-a39a-76831ab3b832 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:50.070995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-18b2892f-e332-4515-a39a-76831ab3b832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:50.141016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 572/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:50.403453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa670167-3c54-472f-a987-639f41c877a4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:50.491062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:50.527897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa670167-3c54-472f-a987-639f41c877a4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:50.572152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bfe00-ff4d-4e46-b245-4f569b912d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:50.573069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bfe00-ff4d-4e46-b245-4f569b912d86 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:50.573507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bfe00-ff4d-4e46-b245-4f569b912d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:50.573929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bfe00-ff4d-4e46-b245-4f569b912d86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:50.649889 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bfe00-ff4d-4e46-b245-4f569b912d86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:50.651550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bfe00-ff4d-4e46-b245-4f569b912d86 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:50.739762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:50.773460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 592/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:23:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:23:50.779685 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2892f-e332-4515-a39a-76831ab3b832 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:50.831663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:50.886965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2892f-e332-4515-a39a-76831ab3b832 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:50.921713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3add18c-9023-4a60-87da-85f474aad5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:50.922328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add18c-9023-4a60-87da-85f474aad5cc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:50.922603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add18c-9023-4a60-87da-85f474aad5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:50.922904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add18c-9023-4a60-87da-85f474aad5cc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:50.930502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3add18c-9023-4a60-87da-85f474aad5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:50.931966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3add18c-9023-4a60-87da-85f474aad5cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f3add18c-9023-4a60-87da-85f474aad5cc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:23:50.944818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:23:50.946393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:23:50.947526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 573/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:23:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:23:50.955137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bfe00-ff4d-4e46-b245-4f569b912d86 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:51.069056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bfe00-ff4d-4e46-b245-4f569b912d86 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:51.105856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62046a5a-05d1-46f5-a321-360464a173a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:51.106465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62046a5a-05d1-46f5-a321-360464a173a2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:51.106766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62046a5a-05d1-46f5-a321-360464a173a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:51.107041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62046a5a-05d1-46f5-a321-360464a173a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:51.107690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62046a5a-05d1-46f5-a321-360464a173a2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:23:51.107964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62046a5a-05d1-46f5-a321-360464a173a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:51.109362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62046a5a-05d1-46f5-a321-360464a173a2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:51.117096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 593/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Aug 31 04:23:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:52.983729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf479f5-454e-4896-a6fe-bcce18a28913 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:52.984355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf479f5-454e-4896-a6fe-bcce18a28913 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:52.984355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf479f5-454e-4896-a6fe-bcce18a28913 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:52.985633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 574/1167] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:52.992028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df9785-e4a5-4795-823d-1781935ca1ea None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:52.992758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df9785-e4a5-4795-823d-1781935ca1ea None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:52.992910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df9785-e4a5-4795-823d-1781935ca1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:53.022276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c0df9785-e4a5-4795-823d-1781935ca1ea None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:53.023492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c0df9785-e4a5-4795-823d-1781935ca1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:53.054307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:53.099866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 594/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:53.364454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf479f5-454e-4896-a6fe-bcce18a28913 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:53.488202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf479f5-454e-4896-a6fe-bcce18a28913 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:53.497258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:53.529211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e936af-6bec-455d-8ec0-0736e11a8015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:53.529811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e936af-6bec-455d-8ec0-0736e11a8015 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:53.530086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e936af-6bec-455d-8ec0-0736e11a8015 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:53.530355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e936af-6bec-455d-8ec0-0736e11a8015 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:53.531065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e936af-6bec-455d-8ec0-0736e11a8015 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:53.532435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e936af-6bec-455d-8ec0-0736e11a8015 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:53.552718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 575/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Wed Aug 31 04:23:53 2022] POST /identity/v3/domains => generated 267 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:23:55.446118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3991ea-461b-48fa-98b3-d9e6a57f126d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:55.446667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3991ea-461b-48fa-98b3-d9e6a57f126d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:55.446667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3991ea-461b-48fa-98b3-d9e6a57f126d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:55.448001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 595/1170] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:55.453557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715d34-cbfe-409b-bfce-18046c12a5b7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:55.453988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715d34-cbfe-409b-bfce-18046c12a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:55.454081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715d34-cbfe-409b-bfce-18046c12a5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:55.477594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8f715d34-cbfe-409b-bfce-18046c12a5b7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:55.478820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f715d34-cbfe-409b-bfce-18046c12a5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:55.545697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 576/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:55.815502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3991ea-461b-48fa-98b3-d9e6a57f126d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:55.937350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3991ea-461b-48fa-98b3-d9e6a57f126d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:55.973467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5b021-12aa-43f2-9ead-a108a237ebdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:55.974088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b021-12aa-43f2-9ead-a108a237ebdc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:55.974354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b021-12aa-43f2-9ead-a108a237ebdc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:55.974619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5b021-12aa-43f2-9ead-a108a237ebdc None admin] PATH_INFO: `/v3/domains/437c47b73f2d44ef92d9f7f83d13aa85` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:55.986375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5b021-12aa-43f2-9ead-a108a237ebdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=437c47b73f2d44ef92d9f7f83d13aa85)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:55.987865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5b021-12aa-43f2-9ead-a108a237ebdc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:55.989437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 596/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Aug 31 04:23:55 2022] GET /identity/v3/domains/437c47b73f2d44ef92d9f7f83d13aa85 => generated 267 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:55.995599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f715d34-cbfe-409b-bfce-18046c12a5b7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:56.126403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:56.130994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f715d34-cbfe-409b-bfce-18046c12a5b7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:56.166090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3febb-3d36-4da2-a5cd-49c93fa58b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:56.166699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3febb-3d36-4da2-a5cd-49c93fa58b77 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:56.166949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3febb-3d36-4da2-a5cd-49c93fa58b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:56.167261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3febb-3d36-4da2-a5cd-49c93fa58b77 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:56.167891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3febb-3d36-4da2-a5cd-49c93fa58b77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:56.169311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3febb-3d36-4da2-a5cd-49c93fa58b77 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:56.198910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 577/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:23:55 2022] POST /identity/v3/projects => generated 361 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:23:58.113466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bf668-a971-449f-9c5d-a6ae2bab4f00 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:58.114239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bf668-a971-449f-9c5d-a6ae2bab4f00 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:58.114429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bf668-a971-449f-9c5d-a6ae2bab4f00 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:58.116693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 597/1174] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:23:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:23:58.123169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d926f-96c1-4cf7-8db7-b25a50de78cd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:58.123620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d926f-96c1-4cf7-8db7-b25a50de78cd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:58.123721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d926f-96c1-4cf7-8db7-b25a50de78cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:58.149913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2a6d926f-96c1-4cf7-8db7-b25a50de78cd None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:23:58.151211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a6d926f-96c1-4cf7-8db7-b25a50de78cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:58.222238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 578/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:23:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:23:58.484084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bf668-a971-449f-9c5d-a6ae2bab4f00 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:58.519738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:58.598894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bf668-a971-449f-9c5d-a6ae2bab4f00 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:58.633285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8c908-dec8-4a64-b742-12bc4dba4a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:58.633903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8c908-dec8-4a64-b742-12bc4dba4a5d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:58.634173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8c908-dec8-4a64-b742-12bc4dba4a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:58.634446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8c908-dec8-4a64-b742-12bc4dba4a5d None admin] PATH_INFO: `/v3/domains/437c47b73f2d44ef92d9f7f83d13aa85` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:58.635732 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8c908-dec8-4a64-b742-12bc4dba4a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=437c47b73f2d44ef92d9f7f83d13aa85)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:58.637186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8c908-dec8-4a64-b742-12bc4dba4a5d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:58.638748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 598/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Aug 31 04:23:58 2022] GET /identity/v3/domains/437c47b73f2d44ef92d9f7f83d13aa85 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:23:58.647423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d926f-96c1-4cf7-8db7-b25a50de78cd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:58.732766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:58.754162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d926f-96c1-4cf7-8db7-b25a50de78cd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:58.791311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7b44c-7fd4-47e6-b369-716be277fae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:58.791913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7b44c-7fd4-47e6-b369-716be277fae5 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:58.792191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7b44c-7fd4-47e6-b369-716be277fae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:58.792456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7b44c-7fd4-47e6-b369-716be277fae5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:58.793079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7b44c-7fd4-47e6-b369-716be277fae5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:58.794506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7b44c-7fd4-47e6-b369-716be277fae5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:58.819374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 579/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:23:58 2022] POST /identity/v3/users => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:24:00.740358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ab1fb-c195-44ce-acb2-47329c085b40 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:00.740922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ab1fb-c195-44ce-acb2-47329c085b40 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:00.740922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ab1fb-c195-44ce-acb2-47329c085b40 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:00.742269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 599/1178] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:24:00.747825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881eeb2-ce32-46c3-9e3a-5455d0d3a768 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:00.748258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881eeb2-ce32-46c3-9e3a-5455d0d3a768 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:00.748354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881eeb2-ce32-46c3-9e3a-5455d0d3a768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:00.769137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b881eeb2-ce32-46c3-9e3a-5455d0d3a768 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:24:00.770305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b881eeb2-ce32-46c3-9e3a-5455d0d3a768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:24:00.852180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 580/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:24:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:24:01.122674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ab1fb-c195-44ce-acb2-47329c085b40 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:01.244482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:01.257706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ab1fb-c195-44ce-acb2-47329c085b40 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:01.298895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6092e-666c-4099-a35a-127ec9d6e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:01.299590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6092e-666c-4099-a35a-127ec9d6e1cd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:01.299862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6092e-666c-4099-a35a-127ec9d6e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:01.300131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6092e-666c-4099-a35a-127ec9d6e1cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:01.310926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6092e-666c-4099-a35a-127ec9d6e1cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-99e6092e-666c-4099-a35a-127ec9d6e1cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:24:01.315805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:24:01.316919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:24:01.318121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:24:01.322265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 600/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:24:01.323486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881eeb2-ce32-46c3-9e3a-5455d0d3a768 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:01.390102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:01.425340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881eeb2-ce32-46c3-9e3a-5455d0d3a768 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:01.459968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:01.460561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:01.460833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:01.461095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:01.461752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:24:01.462052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:01.463456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:01.471865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 581/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:24:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:24:01.477953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6092e-666c-4099-a35a-127ec9d6e1cd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:01.591511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6092e-666c-4099-a35a-127ec9d6e1cd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:01.626759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f621d5-8453-4655-abb7-b107fe8265ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:01.627433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f621d5-8453-4655-abb7-b107fe8265ae None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:01.627714 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f621d5-8453-4655-abb7-b107fe8265ae None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:01.627985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f621d5-8453-4655-abb7-b107fe8265ae None admin] PATH_INFO: `/v3/users/da8f167c48bc40a2998da5ffbafb5def` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:01.641720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f621d5-8453-4655-abb7-b107fe8265ae None admin] RBAC: Authorizing `identity:get_user(user_id=da8f167c48bc40a2998da5ffbafb5def)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:01.643252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f621d5-8453-4655-abb7-b107fe8265ae None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:01.644849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 601/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:24:01 2022] GET /identity/v3/users/da8f167c48bc40a2998da5ffbafb5def => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:01.653782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:01.763095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694060a5-efb7-48cf-9610-86e25a3dee03 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:01.789378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:01.799333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa9ce6-cb5f-4682-a9af-2528087be92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:01.800033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa9ce6-cb5f-4682-a9af-2528087be92f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:01.800390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa9ce6-cb5f-4682-a9af-2528087be92f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:01.800663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa9ce6-cb5f-4682-a9af-2528087be92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:01.897589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa9ce6-cb5f-4682-a9af-2528087be92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:01.898983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa9ce6-cb5f-4682-a9af-2528087be92f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:02.000491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 582/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:24:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 04:24:02.006957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f621d5-8453-4655-abb7-b107fe8265ae None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:02.023099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:02.115244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f621d5-8453-4655-abb7-b107fe8265ae None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:02.152480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8228a2-4acb-4a0f-af6f-38177070965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:02.153032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8228a2-4acb-4a0f-af6f-38177070965b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:02.153306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8228a2-4acb-4a0f-af6f-38177070965b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:02.153577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8228a2-4acb-4a0f-af6f-38177070965b None admin] PATH_INFO: `/v3/projects/e4a948bd6d8b4599b00408c57abfe9c7` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:02.154853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8228a2-4acb-4a0f-af6f-38177070965b None admin] RBAC: Authorizing `identity:get_project(project_id=e4a948bd6d8b4599b00408c57abfe9c7)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:02.156304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8228a2-4acb-4a0f-af6f-38177070965b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:02.157946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 602/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:24:02 2022] GET /identity/v3/projects/e4a948bd6d8b4599b00408c57abfe9c7 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:24:02.168502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa9ce6-cb5f-4682-a9af-2528087be92f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:02.206128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:02.286469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa9ce6-cb5f-4682-a9af-2528087be92f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:02.321465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724b071-9c9c-4f9d-a80c-061038d5394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:02.322060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b071-9c9c-4f9d-a80c-061038d5394f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:02.322334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b071-9c9c-4f9d-a80c-061038d5394f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:02.322601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b071-9c9c-4f9d-a80c-061038d5394f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:02.323356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724b071-9c9c-4f9d-a80c-061038d5394f None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=e4a948bd6d8b4599b00408c57abfe9c7, user.id=da8f167c48bc40a2998da5ffbafb5def) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:24:02.323661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724b071-9c9c-4f9d-a80c-061038d5394f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:02.325084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724b071-9c9c-4f9d-a80c-061038d5394f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:02.330553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 583/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:24:02 2022] GET /identity/v3/role_assignments?user.id=da8f167c48bc40a2998da5ffbafb5def&scope.project.id=e4a948bd6d8b4599b00408c57abfe9c7&role.id=241e99d365564c0fa95cb712f4004785 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:04.209210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978018f2-34ab-4534-86dc-e07b6445f18c None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:04.210033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978018f2-34ab-4534-86dc-e07b6445f18c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:04.210033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978018f2-34ab-4534-86dc-e07b6445f18c None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:04.212227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 603/1186] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:24:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:24:04.217637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2646f2-6001-4c6a-b57d-45dc29539e41 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:04.218075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2646f2-6001-4c6a-b57d-45dc29539e41 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:04.218167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2646f2-6001-4c6a-b57d-45dc29539e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:04.243407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8b2646f2-6001-4c6a-b57d-45dc29539e41 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:24:04.244576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8b2646f2-6001-4c6a-b57d-45dc29539e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:24:04.314758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 584/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:24:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:24:04.576599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978018f2-34ab-4534-86dc-e07b6445f18c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:04.661838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:04.692296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978018f2-34ab-4534-86dc-e07b6445f18c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:04.737736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e402d5d-feaa-4b0d-a1b7-17548754658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:04.738365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402d5d-feaa-4b0d-a1b7-17548754658c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:04.738647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402d5d-feaa-4b0d-a1b7-17548754658c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:04.738947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402d5d-feaa-4b0d-a1b7-17548754658c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:04.750759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e402d5d-feaa-4b0d-a1b7-17548754658c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0e402d5d-feaa-4b0d-a1b7-17548754658c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:24:04.755623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:24:04.756675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:24:04.757614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 604/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:24:04.763004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2646f2-6001-4c6a-b57d-45dc29539e41 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:04.787267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:04.868065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2646f2-6001-4c6a-b57d-45dc29539e41 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:04.903269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:04.903876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:04.904137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:04.904404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:04.905065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:24:04.905363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:04.906753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:04.915457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 585/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:24:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:04.923095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e402d5d-feaa-4b0d-a1b7-17548754658c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:05.028769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e402d5d-feaa-4b0d-a1b7-17548754658c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:05.069236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637dd4c-3088-400f-b6b0-6a856fb2e494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:05.070168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dd4c-3088-400f-b6b0-6a856fb2e494 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:05.070572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dd4c-3088-400f-b6b0-6a856fb2e494 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:05.071355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dd4c-3088-400f-b6b0-6a856fb2e494 None admin] PATH_INFO: `/v3/users/da8f167c48bc40a2998da5ffbafb5def` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:05.073301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637dd4c-3088-400f-b6b0-6a856fb2e494 None admin] RBAC: Authorizing `identity:get_user(user_id=da8f167c48bc40a2998da5ffbafb5def)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:05.075679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637dd4c-3088-400f-b6b0-6a856fb2e494 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:05.078108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 605/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:24:04 2022] GET /identity/v3/users/da8f167c48bc40a2998da5ffbafb5def => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:05.085212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:05.176726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:05.194036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e2370-29fa-4498-b7b1-6a41e60dbb32 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:05.229324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdc7db-54cd-4a77-9166-d7e456dc246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:05.229939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc7db-54cd-4a77-9166-d7e456dc246e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:05.230192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc7db-54cd-4a77-9166-d7e456dc246e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:05.230459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc7db-54cd-4a77-9166-d7e456dc246e None admin] PATH_INFO: `/v3/projects/e4a948bd6d8b4599b00408c57abfe9c7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:05.231706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdc7db-54cd-4a77-9166-d7e456dc246e None admin] RBAC: Authorizing `identity:get_project(project_id=e4a948bd6d8b4599b00408c57abfe9c7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:05.233137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdc7db-54cd-4a77-9166-d7e456dc246e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:05.234760 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 586/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:24:05 2022] GET /identity/v3/projects/e4a948bd6d8b4599b00408c57abfe9c7 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:05.241122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637dd4c-3088-400f-b6b0-6a856fb2e494 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:05.252869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:05.357876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637dd4c-3088-400f-b6b0-6a856fb2e494 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:05.403779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e4b90-4ec6-47d4-8427-fb17e1957537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:05.404805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e4b90-4ec6-47d4-8427-fb17e1957537 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:05.405269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e4b90-4ec6-47d4-8427-fb17e1957537 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:05.405703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e4b90-4ec6-47d4-8427-fb17e1957537 None admin] PATH_INFO: `/v3/projects/e4a948bd6d8b4599b00408c57abfe9c7/users/da8f167c48bc40a2998da5ffbafb5def/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:05.409284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e4b90-4ec6-47d4-8427-fb17e1957537 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a948bd6d8b4599b00408c57abfe9c7, user_id=da8f167c48bc40a2998da5ffbafb5def, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:05.411867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e4b90-4ec6-47d4-8427-fb17e1957537 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:05.430049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 606/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:24:05 2022] PUT /identity/v3/projects/e4a948bd6d8b4599b00408c57abfe9c7/users/da8f167c48bc40a2998da5ffbafb5def/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:24:07.286772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e31b4-e567-4beb-acaa-d75bc810d684 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:07.287604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e31b4-e567-4beb-acaa-d75bc810d684 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:07.287604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e31b4-e567-4beb-acaa-d75bc810d684 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:07.289697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 587/1193] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:24:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:24:07.295055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c834-33d0-40f6-84f7-289d1f64ec9e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:07.295491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c834-33d0-40f6-84f7-289d1f64ec9e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:07.295590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19c834-33d0-40f6-84f7-289d1f64ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:07.324047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9a19c834-33d0-40f6-84f7-289d1f64ec9e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:24:07.325858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9a19c834-33d0-40f6-84f7-289d1f64ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:24:07.396180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:07.402764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 607/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:24:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:24:07.666209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e31b4-e567-4beb-acaa-d75bc810d684 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:07.797919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e31b4-e567-4beb-acaa-d75bc810d684 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:07.837253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fdb73-6fba-4f86-8054-96d41ceb51c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:07.837869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fdb73-6fba-4f86-8054-96d41ceb51c1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:07.838153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fdb73-6fba-4f86-8054-96d41ceb51c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:07.838426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fdb73-6fba-4f86-8054-96d41ceb51c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:07.849684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fdb73-6fba-4f86-8054-96d41ceb51c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-520fdb73-6fba-4f86-8054-96d41ceb51c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:24:07.854495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:24:07.855563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:24:07.856532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 588/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:24:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:24:07.861928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19c834-33d0-40f6-84f7-289d1f64ec9e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:07.978935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19c834-33d0-40f6-84f7-289d1f64ec9e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:08.020015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:08.021162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:08.021451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:08.021722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:08.022391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:24:08.022694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:08.024098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:08.032339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 608/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:24:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:08.042678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fdb73-6fba-4f86-8054-96d41ceb51c1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:08.062018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:08.154130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fdb73-6fba-4f86-8054-96d41ceb51c1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:08.190162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ae390-ab65-40bd-810e-ea5823f7b109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:08.190775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae390-ab65-40bd-810e-ea5823f7b109 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:08.191081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae390-ab65-40bd-810e-ea5823f7b109 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:08.191349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae390-ab65-40bd-810e-ea5823f7b109 None admin] PATH_INFO: `/v3/users/da8f167c48bc40a2998da5ffbafb5def` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:08.192670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ae390-ab65-40bd-810e-ea5823f7b109 None admin] RBAC: Authorizing `identity:get_user(user_id=da8f167c48bc40a2998da5ffbafb5def)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:08.194410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ae390-ab65-40bd-810e-ea5823f7b109 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:08.196400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 589/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Aug 31 04:24:08 2022] GET /identity/v3/users/da8f167c48bc40a2998da5ffbafb5def => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:08.206202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:08.274690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:08.326206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59408fb-0fd5-41f6-996d-d76a3fc66b17 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:08.363853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc557-f765-4a94-9cdd-3f3e1e0a4e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:08.364454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc557-f765-4a94-9cdd-3f3e1e0a4e88 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:08.364751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc557-f765-4a94-9cdd-3f3e1e0a4e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:08.365017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc557-f765-4a94-9cdd-3f3e1e0a4e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:08.444057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc557-f765-4a94-9cdd-3f3e1e0a4e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:08.445504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc557-f765-4a94-9cdd-3f3e1e0a4e88 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:08.503191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:08.551249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 609/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:24:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:24:08.558524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ae390-ab65-40bd-810e-ea5823f7b109 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:08.657562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:08.678039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ae390-ab65-40bd-810e-ea5823f7b109 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:08.712603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4a1eb-023a-4efe-bcb1-c6706d0391fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:08.713215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a1eb-023a-4efe-bcb1-c6706d0391fa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:08.713492 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a1eb-023a-4efe-bcb1-c6706d0391fa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:08.713758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a1eb-023a-4efe-bcb1-c6706d0391fa None admin] PATH_INFO: `/v3/projects/e4a948bd6d8b4599b00408c57abfe9c7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:08.714879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4a1eb-023a-4efe-bcb1-c6706d0391fa None admin] RBAC: Authorizing `identity:get_project(project_id=e4a948bd6d8b4599b00408c57abfe9c7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:08.716319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4a1eb-023a-4efe-bcb1-c6706d0391fa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:08.717948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 590/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Aug 31 04:24:08 2022] GET /identity/v3/projects/e4a948bd6d8b4599b00408c57abfe9c7 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:08.724495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc557-f765-4a94-9cdd-3f3e1e0a4e88 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:24:08.836147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc557-f765-4a94-9cdd-3f3e1e0a4e88 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:24:08.871453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7c941-1c46-403d-8d98-b8006114a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:24:08.872039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c941-1c46-403d-8d98-b8006114a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:24:08.872311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c941-1c46-403d-8d98-b8006114a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:24:08.872609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c941-1c46-403d-8d98-b8006114a0d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:24:08.873322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7c941-1c46-403d-8d98-b8006114a0d8 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=e4a948bd6d8b4599b00408c57abfe9c7, user.id=da8f167c48bc40a2998da5ffbafb5def) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:24:08.873632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7c941-1c46-403d-8d98-b8006114a0d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:24:08.875090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7c941-1c46-403d-8d98-b8006114a0d8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:24:08.880887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 610/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:24:08 2022] GET /identity/v3/role_assignments?user.id=da8f167c48bc40a2998da5ffbafb5def&scope.project.id=e4a948bd6d8b4599b00408c57abfe9c7&role.id=241e99d365564c0fa95cb712f4004785 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:24:18.708111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=26 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:18.878280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:04.754485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868eb9c-f128-48aa-bf27-29e275eac978 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:04.755288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868eb9c-f128-48aa-bf27-29e275eac978 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:04.755288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868eb9c-f128-48aa-bf27-29e275eac978 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:04.756406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 591/1201] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:04.762524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f00378-584c-4823-82a9-bb591d0463b8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:04.763126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f00378-584c-4823-82a9-bb591d0463b8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:04.763221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f00378-584c-4823-82a9-bb591d0463b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:04.784103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-38f00378-584c-4823-82a9-bb591d0463b8 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:04.785456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-38f00378-584c-4823-82a9-bb591d0463b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:04.884465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 611/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:05.178119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868eb9c-f128-48aa-bf27-29e275eac978 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:05.318783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868eb9c-f128-48aa-bf27-29e275eac978 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:05.369737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:05.371005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:05.371596 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:05.371952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:05.385402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:05.387247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:05.408204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:05.409387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:05.415950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:05.415950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:05.415950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:05.415950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 592/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:05.416516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f00378-584c-4823-82a9-bb591d0463b8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:05.549052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f00378-584c-4823-82a9-bb591d0463b8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:05.586252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:05.593059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb2f7a-3094-47c7-bd2a-b9bf2f675784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:05.593842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb2f7a-3094-47c7-bd2a-b9bf2f675784 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:05.594143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb2f7a-3094-47c7-bd2a-b9bf2f675784 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:05.594381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb2f7a-3094-47c7-bd2a-b9bf2f675784 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:05.595254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb2f7a-3094-47c7-bd2a-b9bf2f675784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:05.595380 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb2f7a-3094-47c7-bd2a-b9bf2f675784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:05.596880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb2f7a-3094-47c7-bd2a-b9bf2f675784 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:05.614164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 612/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:05.615564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:05.693053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:05.752749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d80b56-a1cd-45fe-9c79-d86ee8a1fcaf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:05.797011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ac25f-3ac0-4aa3-a9ae-edf139a94664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:05.797859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ac25f-3ac0-4aa3-a9ae-edf139a94664 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:05.798230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ac25f-3ac0-4aa3-a9ae-edf139a94664 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:05.798772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ac25f-3ac0-4aa3-a9ae-edf139a94664 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:05.799577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ac25f-3ac0-4aa3-a9ae-edf139a94664 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:05.801171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ac25f-3ac0-4aa3-a9ae-edf139a94664 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:05.834744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 593/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Aug 31 04:25:05 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:25:07.995309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a184-c47b-42f5-93cc-bf4c6083ce64 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:07.995959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a184-c47b-42f5-93cc-bf4c6083ce64 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:07.995959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a184-c47b-42f5-93cc-bf4c6083ce64 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:07.997439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 613/1206] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:08.003779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c0bb4-8646-4801-9858-b294aff898d9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:08.004250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c0bb4-8646-4801-9858-b294aff898d9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:08.004415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c0bb4-8646-4801-9858-b294aff898d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:08.025252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6c2c0bb4-8646-4801-9858-b294aff898d9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:08.026544 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c2c0bb4-8646-4801-9858-b294aff898d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:08.104667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 594/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:08.389979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a184-c47b-42f5-93cc-bf4c6083ce64 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:08.514178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a184-c47b-42f5-93cc-bf4c6083ce64 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:08.554468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43de0f5-884f-4add-b8b6-2da4880ad215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:08.555268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43de0f5-884f-4add-b8b6-2da4880ad215 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:08.555619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43de0f5-884f-4add-b8b6-2da4880ad215 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:08.556329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43de0f5-884f-4add-b8b6-2da4880ad215 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:08.569890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43de0f5-884f-4add-b8b6-2da4880ad215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b43de0f5-884f-4add-b8b6-2da4880ad215 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:25:08.576010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:08.577300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:08.578179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 614/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:25:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:08.583451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c0bb4-8646-4801-9858-b294aff898d9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:08.631020 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:08.714671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c0bb4-8646-4801-9858-b294aff898d9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:08.758145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:08.758824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:08.759118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:08.759459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:08.760137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:08.760473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:08.762050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:08.776247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 595/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:25:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:08.777637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43de0f5-884f-4add-b8b6-2da4880ad215 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:08.833922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:08.890980 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43de0f5-884f-4add-b8b6-2da4880ad215 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:08.928582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:08.929335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:08.929628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:08.929892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:08.941973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:08.943731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:08.959708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:08.960821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:08.961755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:08.961755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:08.961755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:08.961755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 615/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:25:08.967554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:09.084502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:09.087979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b31172-fa25-42dc-9d8c-508e5807518f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:09.127463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:09.128079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:09.128357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:09.128623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:09.129275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:09.129576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:09.130977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:09.147392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 596/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:09.153042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:09.270964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb073d6-1673-4bdd-a98c-65d575892fc9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:09.272966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:09.315848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:09.316480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:09.316757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:09.317031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:09.325510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:09.327000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:25:09.339141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:09.340193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:09.341164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 616/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:09.346366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:09.467051 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5cc54-1680-4dc2-bf23-559f5b6741b2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:09.509595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:09.510277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:09.510622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:09.511017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:09.511800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:09.512075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:09.513708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:09.534959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 597/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 31 04:25:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:09.540615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:09.653508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641402-397d-4598-b8c0-8c173d932fe2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:09.689212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fc520-834c-4393-a6bc-a2b881ee8fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:09.690113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fc520-834c-4393-a6bc-a2b881ee8fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:09.693483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fc520-834c-4393-a6bc-a2b881ee8fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:09.693483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fc520-834c-4393-a6bc-a2b881ee8fd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:09.730987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:09.775539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fc520-834c-4393-a6bc-a2b881ee8fd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:09.777061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fc520-834c-4393-a6bc-a2b881ee8fd4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:09.896855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 617/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:25:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:25:09.903139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:09.923524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:10.014127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b0cf8-0ed3-498c-a950-1e95533dbbe3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:10.051502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:10.052165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:10.052413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:10.052677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:10.064588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:10.065173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:10.081057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:10.082009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:10.082923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:10.082923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:10.082923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:10.082923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 598/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:25:10.088164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fc520-834c-4393-a6bc-a2b881ee8fd4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:10.159609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:10.202463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fc520-834c-4393-a6bc-a2b881ee8fd4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:10.241904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:10.242519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:10.242843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:10.243225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:10.243878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:10.244179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:10.245560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:10.262167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 618/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:10.263859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:10.344086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:10.377036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3865b-e133-413d-94da-1709cb6baf09 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:10.411869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:10.412419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:10.412732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:10.412970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:10.420013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:10.421456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:25:10.432930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:10.438932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:10.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 599/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:25:10.442165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:10.545758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2af08-459d-441e-a235-2e0279afb7ac None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:10.569882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:10.584336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66528d02-570f-45bb-bd34-f882b6f0644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:10.585037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66528d02-570f-45bb-bd34-f882b6f0644d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:10.585380 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66528d02-570f-45bb-bd34-f882b6f0644d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:10.585712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66528d02-570f-45bb-bd34-f882b6f0644d None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:10.586477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66528d02-570f-45bb-bd34-f882b6f0644d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:10.586952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66528d02-570f-45bb-bd34-f882b6f0644d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:10.588474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66528d02-570f-45bb-bd34-f882b6f0644d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:10.602273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 619/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:25:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:10.609524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:10.721397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfccfc5-1616-44b7-b23d-b3acb9d3a3eb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:10.754548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:10.761624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36f96e-270d-4ac0-bcd4-01d55102f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:10.762354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36f96e-270d-4ac0-bcd4-01d55102f230 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:10.762759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36f96e-270d-4ac0-bcd4-01d55102f230 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:10.763310 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36f96e-270d-4ac0-bcd4-01d55102f230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:10.764131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36f96e-270d-4ac0-bcd4-01d55102f230 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=463279871f78480d9fee052518539b94) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:10.764530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36f96e-270d-4ac0-bcd4-01d55102f230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:10.766072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36f96e-270d-4ac0-bcd4-01d55102f230 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:10.772080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 600/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:25:10 2022] GET /identity/v3/role_assignments?user.id=463279871f78480d9fee052518539b94&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:12.817809 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af33eb-bdc7-4f05-ad58-6e12dfd55b4b None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:12.818390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af33eb-bdc7-4f05-ad58-6e12dfd55b4b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:12.818390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af33eb-bdc7-4f05-ad58-6e12dfd55b4b None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:12.819853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 620/1220] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:12.827912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f9211-656f-4448-b315-c71c2abef297 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:12.827912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f9211-656f-4448-b315-c71c2abef297 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:12.827912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f9211-656f-4448-b315-c71c2abef297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:12.849843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ae8f9211-656f-4448-b315-c71c2abef297 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:12.851228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ae8f9211-656f-4448-b315-c71c2abef297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:12.931243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 601/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:13.202458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af33eb-bdc7-4f05-ad58-6e12dfd55b4b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:13.332658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af33eb-bdc7-4f05-ad58-6e12dfd55b4b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:13.371459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846821d8-597f-4e0c-a504-bfab563ceeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:13.372328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846821d8-597f-4e0c-a504-bfab563ceeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:13.372616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846821d8-597f-4e0c-a504-bfab563ceeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:13.372966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846821d8-597f-4e0c-a504-bfab563ceeb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:13.385883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846821d8-597f-4e0c-a504-bfab563ceeb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-846821d8-597f-4e0c-a504-bfab563ceeb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:25:13.391946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:13.393149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:13.394391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 621/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:25:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:13.400521 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f9211-656f-4448-b315-c71c2abef297 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:13.508130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f9211-656f-4448-b315-c71c2abef297 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:13.534583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:13.546737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:13.547390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:13.547668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:13.547958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:13.548720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:13.549058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:13.550643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:13.559610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 602/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:25:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:13.565451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846821d8-597f-4e0c-a504-bfab563ceeb3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:13.610619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:13.671373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846821d8-597f-4e0c-a504-bfab563ceeb3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:13.710824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:13.711482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:13.711758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:13.712123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:13.722588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:13.724254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:13.742271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:13.743489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:13.744566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:13.744566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:13.744566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:13.744566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 622/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:13.750115 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:13.857796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe914-8289-46f6-bd23-a8bd836fd6ee None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:13.897523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:13.898062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:13.898335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:13.898600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:13.899267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:13.899746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:13.901111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:13.913233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 603/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:13.920251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:14.018818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:14.026122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c7ea3-dbf2-4926-85c8-c2cf2ea3fb2c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:14.063391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:14.064031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:14.064309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:14.064601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:14.073820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:14.075403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:25:14.088469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:14.089572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:14.090674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 623/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:14.096118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:14.149419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:14.206678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2a3c4-8b41-42d6-a870-8f1a36efacbe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:14.248292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:14.249040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:14.249364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:14.249705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:14.250443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:14.250787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:14.252346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:14.271891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 604/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 31 04:25:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:14.278343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:14.399706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d85eb-d774-4482-9a5a-4d16ab34befd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:14.442595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:14.443363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:14.443679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:14.444018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:14.456444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:14.458450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:14.474795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:14.475988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:14.477027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:14.477027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:14.477027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:14.477027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 624/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 04:25:14.482236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:14.591377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724adf4-c19b-4ca3-ae41-6155d81d64f9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:14.593910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:14.627700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:14.628394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:14.628705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:14.629110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:14.629835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:14.630139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:14.631796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:14.643827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 605/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:14.649798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:14.660817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:14.768189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fd2a2-9dbe-4f2f-866b-cb370ef765a1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:14.807820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:14.808495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:14.808770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:14.809040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:14.816914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:14.818615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:25:14.832113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:14.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:14.834149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 625/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:25:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:14.839753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:14.950908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc7cad-7de3-4d24-98fe-49628d4b8b72 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:14.984972 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d965b8-cfac-4514-94f1-330ae4b55a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:14.985743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d965b8-cfac-4514-94f1-330ae4b55a3c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:14.986026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d965b8-cfac-4514-94f1-330ae4b55a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:14.986302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d965b8-cfac-4514-94f1-330ae4b55a3c None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:14.987103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d965b8-cfac-4514-94f1-330ae4b55a3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:14.987407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d965b8-cfac-4514-94f1-330ae4b55a3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:14.988846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d965b8-cfac-4514-94f1-330ae4b55a3c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:15.003587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 606/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:15.011824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:15.094128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:15.130208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198170cd-aff7-4d27-862b-5a0766a4164d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:15.167767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95ec81-20f6-4f2d-828b-f4c4477f9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:15.168491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95ec81-20f6-4f2d-828b-f4c4477f9c13 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:15.168799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95ec81-20f6-4f2d-828b-f4c4477f9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:15.169173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95ec81-20f6-4f2d-828b-f4c4477f9c13 None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/463279871f78480d9fee052518539b94/roles/efe57f5de58341b6898d0d4f45a9ffc0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:15.184681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95ec81-20f6-4f2d-828b-f4c4477f9c13 None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=463279871f78480d9fee052518539b94, role_id=efe57f5de58341b6898d0d4f45a9ffc0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:15.186369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95ec81-20f6-4f2d-828b-f4c4477f9c13 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:15.201188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 626/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:25:15 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/463279871f78480d9fee052518539b94/roles/efe57f5de58341b6898d0d4f45a9ffc0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:25:17.112566 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f393548-f9e9-4d3c-9b57-bcc4b5a46a08 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:17.113199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f393548-f9e9-4d3c-9b57-bcc4b5a46a08 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:17.113199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f393548-f9e9-4d3c-9b57-bcc4b5a46a08 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:17.114444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 607/1233] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:17.120063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987364d-4acc-42a4-ad2f-2f716f167803 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:17.120537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987364d-4acc-42a4-ad2f-2f716f167803 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:17.120627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987364d-4acc-42a4-ad2f-2f716f167803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:17.139512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f987364d-4acc-42a4-ad2f-2f716f167803 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:17.140706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f987364d-4acc-42a4-ad2f-2f716f167803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:17.211820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 627/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:17.487198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f393548-f9e9-4d3c-9b57-bcc4b5a46a08 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:17.513710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:17.598632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f393548-f9e9-4d3c-9b57-bcc4b5a46a08 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:17.635896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa49d7c-c861-4ea8-8975-9cf788e8109a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:17.636539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49d7c-c861-4ea8-8975-9cf788e8109a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:17.636817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49d7c-c861-4ea8-8975-9cf788e8109a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:17.637082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49d7c-c861-4ea8-8975-9cf788e8109a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:17.648277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa49d7c-c861-4ea8-8975-9cf788e8109a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-aaa49d7c-c861-4ea8-8975-9cf788e8109a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:25:17.653956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:17.655140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:17.656148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 608/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Aug 31 04:25:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:17.662007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987364d-4acc-42a4-ad2f-2f716f167803 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:17.713207 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:17.778957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987364d-4acc-42a4-ad2f-2f716f167803 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:17.814812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:17.815428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:17.815702 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:17.815973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:17.816645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:17.816946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:17.818352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:17.827531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 628/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Aug 31 04:25:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:17.833716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa49d7c-c861-4ea8-8975-9cf788e8109a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:17.929013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:17.947142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa49d7c-c861-4ea8-8975-9cf788e8109a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:17.987914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:17.988472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:17.988745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:17.989012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:18.000114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:18.001564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:18.016070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:18.022945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:18.024192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:18.024192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:18.024192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:18.024192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 609/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:18.024515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:18.135352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd172476-cb38-44a4-b55a-d425096a80dc None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:18.146205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:18.175287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:18.175996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:18.176382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:18.176669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:18.177368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:18.177763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:18.179168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:18.192163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 629/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:18.198671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:18.308676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5aaae-523b-405f-9852-eb4f157c743b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:18.346933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:18.347487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:18.347766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:18.348034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:18.350710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:18.357906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:18.359598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:25:18.371248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:18.372516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:18.373782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:18.379758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 610/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:18.381591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:18.487314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d23578-424c-4b6f-b2f0-61e8a6a2c135 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:18.522492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:18.523153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:18.523429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:18.523700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:18.524448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:18.524939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:18.526559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:18.545397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 630/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 31 04:25:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:18.551390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:18.656525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34f5be-ea13-4282-8d75-933c6e8a75da None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:18.697430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96050776-8cec-4d06-b371-36c38cda2aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:18.698041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96050776-8cec-4d06-b371-36c38cda2aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:18.698788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96050776-8cec-4d06-b371-36c38cda2aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:18.698962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96050776-8cec-4d06-b371-36c38cda2aa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:18.765215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:18.776397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96050776-8cec-4d06-b371-36c38cda2aa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:18.777917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96050776-8cec-4d06-b371-36c38cda2aa0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:18.916732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 611/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:25:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:25:18.922829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:18.954803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:19.033628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ee0c9-eabe-4bd4-8055-6a6a8f6d12df None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:19.070065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:19.070752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:19.071079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:19.071368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:19.083221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:19.084708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:19.099961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:19.101091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:19.102250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:19.102250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:19.102250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:19.102250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 631/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:19.107747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96050776-8cec-4d06-b371-36c38cda2aa0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:19.226421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:19.233497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96050776-8cec-4d06-b371-36c38cda2aa0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:19.272214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:19.272903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:19.273214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:19.273538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:19.274279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:19.274920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:19.276516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:19.288955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 612/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:19.295007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:19.381478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:19.405997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d488f-64f5-4807-b45b-208f03670090 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:19.440934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:19.441557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:19.441832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:19.442166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:19.449474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:19.451180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:25:19.463962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:19.465045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:19.470887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 632/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:25:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:19.471910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:19.588425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e29d33-5206-408f-a787-f8333c2a86e9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:19.624332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb964c5-6f55-428b-8463-0274db99833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:19.624945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb964c5-6f55-428b-8463-0274db99833c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:19.625222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb964c5-6f55-428b-8463-0274db99833c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:19.625476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb964c5-6f55-428b-8463-0274db99833c None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:19.626294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb964c5-6f55-428b-8463-0274db99833c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:19.626569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb964c5-6f55-428b-8463-0274db99833c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:19.628192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb964c5-6f55-428b-8463-0274db99833c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:19.642217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 613/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:25:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:19.649509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:19.758827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b30d5-4efe-4b83-b597-b01b08a0ad5e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:19.793932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c8547-5440-47dc-8ebf-99874b72b215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:19.794748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8547-5440-47dc-8ebf-99874b72b215 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:19.795066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8547-5440-47dc-8ebf-99874b72b215 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:19.795433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8547-5440-47dc-8ebf-99874b72b215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:19.796280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c8547-5440-47dc-8ebf-99874b72b215 None admin] RBAC: Adding query filter params (role.id=efe57f5de58341b6898d0d4f45a9ffc0, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=463279871f78480d9fee052518539b94) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:19.796606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c8547-5440-47dc-8ebf-99874b72b215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:19.798232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c8547-5440-47dc-8ebf-99874b72b215 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:19.804224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 633/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:25:19 2022] GET /identity/v3/role_assignments?user.id=463279871f78480d9fee052518539b94&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=efe57f5de58341b6898d0d4f45a9ffc0 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:21.922063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda1e2b-a9ad-4bbb-ba4e-6adf858aa823 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:21.922620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda1e2b-a9ad-4bbb-ba4e-6adf858aa823 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:21.922620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda1e2b-a9ad-4bbb-ba4e-6adf858aa823 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:21.923988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 614/1247] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:21.932356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9391b-d326-4ac2-8068-44c693d3be83 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:21.932906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9391b-d326-4ac2-8068-44c693d3be83 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:21.933002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9391b-d326-4ac2-8068-44c693d3be83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:21.940177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:21.955636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e2d9391b-d326-4ac2-8068-44c693d3be83 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:21.957017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e2d9391b-d326-4ac2-8068-44c693d3be83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:22.040740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 634/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:22.340007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda1e2b-a9ad-4bbb-ba4e-6adf858aa823 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:22.363069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:22.465635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda1e2b-a9ad-4bbb-ba4e-6adf858aa823 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:22.506946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9ef54-7885-45b0-ae01-f2df9080c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:22.507664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9ef54-7885-45b0-ae01-f2df9080c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:22.507943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9ef54-7885-45b0-ae01-f2df9080c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:22.508203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9ef54-7885-45b0-ae01-f2df9080c8a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:22.521407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9ef54-7885-45b0-ae01-f2df9080c8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-01c9ef54-7885-45b0-ae01-f2df9080c8a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:25:22.527000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:22.528140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:22.528985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 615/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:25:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:22.534683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9391b-d326-4ac2-8068-44c693d3be83 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:22.645253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9391b-d326-4ac2-8068-44c693d3be83 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:22.678615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:22.684809 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:22.685435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:22.685709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:22.685983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:22.686657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:22.687011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:22.688377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:22.696839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 635/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:25:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:22.704044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9ef54-7885-45b0-ae01-f2df9080c8a7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:22.795188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:22.819694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9ef54-7885-45b0-ae01-f2df9080c8a7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:22.856271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:22.858085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:22.858085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:22.858085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:22.871039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:22.872617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:22.888856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:22.890049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:22.891333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:22.891333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:22.891333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:22.891333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 616/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:22.896575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:23.004692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb352aa-687b-4950-b9c0-20d3deed8b9e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:23.044533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:23.045164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:23.045444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:23.045718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:23.046375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:23.046668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:23.048083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:23.066502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 636/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:23.068831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:23.187690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4411c60-bf7b-4314-ae6c-3785b8d8e8fd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:23.216913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:23.226323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:23.227199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:23.227641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:23.227918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:23.238212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:23.240013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:25:23.252847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:23.254022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:23.255030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 617/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:23.261017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:23.322279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:23.374381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350851e-0bca-4759-bee1-c41bf3ec2713 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:23.418397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:23.419143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:23.419520 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:23.419772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:23.420517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:23.420827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:23.422269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:23.444308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 637/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 31 04:25:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:23.450491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:23.562449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2dbe3-dd55-45e8-87e7-07207deaf3d5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:23.601129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea8283-601a-49bc-abe1-a59e3738abb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:23.601746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea8283-601a-49bc-abe1-a59e3738abb2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:23.602020 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea8283-601a-49bc-abe1-a59e3738abb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:23.602287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea8283-601a-49bc-abe1-a59e3738abb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:23.665117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:23.678928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea8283-601a-49bc-abe1-a59e3738abb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:23.680425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea8283-601a-49bc-abe1-a59e3738abb2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:23.787194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 618/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:25:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 04:25:23.793154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:23.907713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd778e4-3426-4178-a7c4-bc16109bdf6b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:23.923703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:23.947797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:23.948428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:23.949330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:23.949600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:23.960467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:23.962362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:23.978335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:23.979442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:23.980594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:23.980594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:23.980594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:23.980594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 638/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:23.985632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea8283-601a-49bc-abe1-a59e3738abb2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:24.081539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:24.095001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea8283-601a-49bc-abe1-a59e3738abb2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:24.132256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:24.132916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:24.133288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:24.133531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:24.134317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:24.134615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:24.136182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:24.151163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 619/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:24.155750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:24.268765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4466da-1ada-493f-bfe9-af8641531a36 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:24.305377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:24.306159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:24.306439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:24.306711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:24.313363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:24.314853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:25:24.327294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:24.328535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:24.329745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 639/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:24.334731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:24.445391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12954d10-0bec-4a07-a0df-390a513988de None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:24.490540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7017ef-680a-4365-8718-f46468c217a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:24.491196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7017ef-680a-4365-8718-f46468c217a0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:24.491471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7017ef-680a-4365-8718-f46468c217a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:24.491736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7017ef-680a-4365-8718-f46468c217a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:24.492430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7017ef-680a-4365-8718-f46468c217a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:24.492731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7017ef-680a-4365-8718-f46468c217a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:24.494215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7017ef-680a-4365-8718-f46468c217a0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:24.506618 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 620/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:25:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:24.513299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:24.542649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:24.626396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b9c11-9241-4cc5-b240-c489ea0795b2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:24.664765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24455918-c6ee-4f4a-98c8-900e6a49c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:24.665688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455918-c6ee-4f4a-98c8-900e6a49c248 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:24.666037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455918-c6ee-4f4a-98c8-900e6a49c248 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:24.666306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455918-c6ee-4f4a-98c8-900e6a49c248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:24.667063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24455918-c6ee-4f4a-98c8-900e6a49c248 None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=463279871f78480d9fee052518539b94) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:24.667365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24455918-c6ee-4f4a-98c8-900e6a49c248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:24.669038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24455918-c6ee-4f4a-98c8-900e6a49c248 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:24.688282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 640/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:25:24 2022] GET /identity/v3/role_assignments?user.id=463279871f78480d9fee052518539b94&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=241e99d365564c0fa95cb712f4004785 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:26.613548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5380b-e0cc-4d8b-915c-26c1b3fa0ac3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:26.614130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5380b-e0cc-4d8b-915c-26c1b3fa0ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:26.614130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5380b-e0cc-4d8b-915c-26c1b3fa0ac3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:26.615471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 621/1261] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:26.621782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c396e-550b-4242-afae-73cbacafa2ea None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:26.622514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c396e-550b-4242-afae-73cbacafa2ea None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:26.622656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c396e-550b-4242-afae-73cbacafa2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:26.652643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8f1c396e-550b-4242-afae-73cbacafa2ea None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:26.653847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8f1c396e-550b-4242-afae-73cbacafa2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:26.725364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 641/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:26.988896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5380b-e0cc-4d8b-915c-26c1b3fa0ac3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:27.006210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:27.100578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5380b-e0cc-4d8b-915c-26c1b3fa0ac3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:27.136121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a44fb-99e6-4ee7-9b28-a0c413d80db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:27.136750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a44fb-99e6-4ee7-9b28-a0c413d80db5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:27.137021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a44fb-99e6-4ee7-9b28-a0c413d80db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:27.137288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a44fb-99e6-4ee7-9b28-a0c413d80db5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:27.147988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a44fb-99e6-4ee7-9b28-a0c413d80db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ea5a44fb-99e6-4ee7-9b28-a0c413d80db5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:25:27.152810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:27.153861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:27.154929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 622/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:25:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:27.160189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c396e-550b-4242-afae-73cbacafa2ea None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:27.186320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:27.269727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c396e-550b-4242-afae-73cbacafa2ea None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:27.307466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:27.308096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:27.308369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:27.308638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:27.309300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:27.309602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:27.311004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:27.319067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 642/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:25:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:27.328685 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a44fb-99e6-4ee7-9b28-a0c413d80db5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:27.410440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:27.445297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a44fb-99e6-4ee7-9b28-a0c413d80db5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:27.483930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:27.484548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:27.484823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:27.485089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:27.496308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:27.497756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:27.513311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:27.514485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:27.515431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:27.515431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:27.515431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:27.515431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 623/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:27.520658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:27.605463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:27.630019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e06b-47b8-4e1a-af4c-61702624cf99 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:27.667100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:27.667704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:27.667982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:27.668253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:27.668888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:27.669186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:27.670563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:27.682909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 643/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:27.691462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:27.808301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfccae-cd49-4465-a6cd-ab27fb451254 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:27.828914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:27.843972 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:27.844593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:27.844865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:27.845136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:27.853566 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:27.855033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:25:27.866288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:27.867379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:27.868549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:27.870877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 624/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:27.874090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:27.995532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6ba40-3f67-4fb8-bf63-3cfa04f2e56b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:28.030788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:28.031389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:28.031659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:28.031924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:28.032647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:28.032941 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:28.034377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:28.043093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:28.051872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 644/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 31 04:25:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:28.059723 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:28.188248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c8530-0002-4bca-a60e-6c17ae24e44e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:28.223477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067981a-0e84-4418-be14-0934353db282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:28.224129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067981a-0e84-4418-be14-0934353db282 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:28.224404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067981a-0e84-4418-be14-0934353db282 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:28.224675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067981a-0e84-4418-be14-0934353db282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:28.235770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067981a-0e84-4418-be14-0934353db282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:28.237216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067981a-0e84-4418-be14-0934353db282 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b067981a-0e84-4418-be14-0934353db282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:28.251700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:28.252748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:28.253648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:28.253648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:28.253648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:28.253648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 625/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:28.258897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:28.365704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3770a9a-b7d9-4adb-8e70-836736c870e5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:28.410287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:28.411167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:28.411419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:28.411576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:28.412179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:28.412472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:28.413850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:28.427484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 645/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:28.433946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067981a-0e84-4418-be14-0934353db282 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:28.470248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:28.546064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067981a-0e84-4418-be14-0934353db282 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:28.588163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:28.588783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:28.589063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:28.589332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:28.596463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:28.597927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:25:28.609762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:28.610734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:28.611785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:25:28.611785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:25:28.611785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:25:28.611785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:25:28.611785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:25:28.611785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:25:28.611785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:28.611785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:28.614894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 626/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:28.616948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:28.679983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:28.726603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b84b7d-63ac-4306-b6ab-c6968d278591 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:28.764191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a92d1-3334-4149-b6c9-ce1436720a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:28.764790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a92d1-3334-4149-b6c9-ce1436720a8b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:28.765069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a92d1-3334-4149-b6c9-ce1436720a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:28.765340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a92d1-3334-4149-b6c9-ce1436720a8b None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:28.766030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a92d1-3334-4149-b6c9-ce1436720a8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:28.766339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a92d1-3334-4149-b6c9-ce1436720a8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:28.767818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a92d1-3334-4149-b6c9-ce1436720a8b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:28.780447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 646/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:25:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:28.787257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:28.894811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:28.904984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d156d73-e1aa-48c0-9348-3d43fabd84cf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:28.941793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a1d7-bd4f-4b19-9b55-1cdf2b05866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:28.942435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a1d7-bd4f-4b19-9b55-1cdf2b05866f None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:28.942682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a1d7-bd4f-4b19-9b55-1cdf2b05866f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:28.943003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a1d7-bd4f-4b19-9b55-1cdf2b05866f None admin] PATH_INFO: `/v3/projects/c15c751e94034d7591fabc8910a7076b/users/463279871f78480d9fee052518539b94/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:28.945127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631a1d7-bd4f-4b19-9b55-1cdf2b05866f None admin] RBAC: Authorizing `identity:create_grant(project_id=c15c751e94034d7591fabc8910a7076b, user_id=463279871f78480d9fee052518539b94, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:28.946555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631a1d7-bd4f-4b19-9b55-1cdf2b05866f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:28.962838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 627/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Aug 31 04:25:28 2022] PUT /identity/v3/projects/c15c751e94034d7591fabc8910a7076b/users/463279871f78480d9fee052518539b94/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:25:30.832784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff040c80-8b29-4a5f-b756-0018e75bd3ab None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:30.833808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff040c80-8b29-4a5f-b756-0018e75bd3ab None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:30.833808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff040c80-8b29-4a5f-b756-0018e75bd3ab None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:30.835816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 647/1274] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:30.844712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66bba-3bf8-41d0-beae-1a450e466f71 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:30.845141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66bba-3bf8-41d0-beae-1a450e466f71 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:30.845271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66bba-3bf8-41d0-beae-1a450e466f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:30.867408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4ba66bba-3bf8-41d0-beae-1a450e466f71 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:30.868580 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ba66bba-3bf8-41d0-beae-1a450e466f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:30.943699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 628/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:31.219039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff040c80-8b29-4a5f-b756-0018e75bd3ab None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:31.338609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff040c80-8b29-4a5f-b756-0018e75bd3ab None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:31.357863 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:31.382502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163281d-0b15-4b88-8222-75e01f64297c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:31.383264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163281d-0b15-4b88-8222-75e01f64297c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:31.383547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163281d-0b15-4b88-8222-75e01f64297c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:31.383887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163281d-0b15-4b88-8222-75e01f64297c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:31.396785 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163281d-0b15-4b88-8222-75e01f64297c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8163281d-0b15-4b88-8222-75e01f64297c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 04:25:31.402106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:31.403641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:31.404501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 648/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Aug 31 04:25:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:31.409935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66bba-3bf8-41d0-beae-1a450e466f71 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:31.496888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:31.516202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66bba-3bf8-41d0-beae-1a450e466f71 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:31.552246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:31.553171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:31.553625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:31.554065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:31.555198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:31.555671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:31.557862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:31.571257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 629/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Aug 31 04:25:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:31.578011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163281d-0b15-4b88-8222-75e01f64297c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:31.682775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163281d-0b15-4b88-8222-75e01f64297c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:31.717573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:31.718189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:31.718458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:31.718746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:31.729846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:31.731336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:31.745789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:31.746885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:31.747996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:31.747996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:31.747996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:31.747996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 649/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:31.753520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:31.860599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba104838-111f-459b-b66c-9a4809b62901 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:31.895784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:31.896376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:31.896649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:31.896919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:31.897561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:31.897851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:31.899262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:31.911014 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 630/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:25:31.917691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:31.955406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:32.034443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b396-b388-4197-8776-975103fe28a6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:32.070242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:32.070896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:32.071171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:32.071420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:32.080367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:32.081806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 04:25:32.092748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:32.093782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:32.094746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 650/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:32.099902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:32.104081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:32.206043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a03ca-cc46-4df8-ae19-1e3226ec1515 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:32.247536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:32.248152 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:32.248431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:32.248698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:32.249430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:32.249724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:32.251193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:32.268468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 631/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Aug 31 04:25:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:32.277749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:32.375034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:32.387636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d410be3-c0c4-443e-a96c-d78dc058ca39 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:32.434617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e7787-7021-46b2-b4d0-647db5ea7fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:32.435246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7787-7021-46b2-b4d0-647db5ea7fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:32.435524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7787-7021-46b2-b4d0-647db5ea7fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:32.435791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7787-7021-46b2-b4d0-647db5ea7fb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:32.506198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e7787-7021-46b2-b4d0-647db5ea7fb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:32.507701 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e7787-7021-46b2-b4d0-647db5ea7fb4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:32.604063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:32.621428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 651/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Aug 31 04:25:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 04:25:32.628126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:32.705972 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:32.748937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e129ac-2378-4b0f-9f2a-4b86f20e087a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:32.784278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:32.784894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:32.785172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:32.785467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:32.795983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:32.797427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 04:25:32.812990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:32.814028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 04:25:32.815183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 04:25:32.815183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 04:25:32.815183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:32.815183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 632/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:32.822763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e7787-7021-46b2-b4d0-647db5ea7fb4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:32.928859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e7787-7021-46b2-b4d0-647db5ea7fb4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:32.965775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:32.966378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:32.966652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:32.966965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:32.967594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:32.967890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:32.969254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:32.980459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 652/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Aug 31 04:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:32.989815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:33.110455 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9acd08-5e01-4ac3-bf25-6eecb03eaf1a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:33.124190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:33.145545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:33.146164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:33.146444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:33.146707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:33.153711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:33.155188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 04:25:33.167334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:33.168326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 31 04:25:33.169533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 633/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Aug 31 04:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:33.174508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:33.230316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:33.281727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378c127-f6e6-4ac4-af90-cb6fde61d6be None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:33.316651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beb2bb-e6a2-4040-9de7-282bbb45970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:33.317242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb2bb-e6a2-4040-9de7-282bbb45970a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:33.317519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb2bb-e6a2-4040-9de7-282bbb45970a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:33.317784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb2bb-e6a2-4040-9de7-282bbb45970a None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:33.318479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beb2bb-e6a2-4040-9de7-282bbb45970a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:33.318805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beb2bb-e6a2-4040-9de7-282bbb45970a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:33.320237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beb2bb-e6a2-4040-9de7-282bbb45970a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:33.332493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 653/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Aug 31 04:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:33.339503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:33.458196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260012a-1fb1-4768-90ef-f23e8a3618e3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:33.492747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019d861-7a4a-4f38-a0b7-f5205df221ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:33.493271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019d861-7a4a-4f38-a0b7-f5205df221ac None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:33.493539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019d861-7a4a-4f38-a0b7-f5205df221ac None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:33.493800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019d861-7a4a-4f38-a0b7-f5205df221ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:33.494568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019d861-7a4a-4f38-a0b7-f5205df221ac None admin] RBAC: Adding query filter params (role.id=241e99d365564c0fa95cb712f4004785, scope.project.id=c15c751e94034d7591fabc8910a7076b, user.id=463279871f78480d9fee052518539b94) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:33.494900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019d861-7a4a-4f38-a0b7-f5205df221ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:33.496328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019d861-7a4a-4f38-a0b7-f5205df221ac None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:33.502110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 634/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Aug 31 04:25:33 2022] GET /identity/v3/role_assignments?user.id=463279871f78480d9fee052518539b94&scope.project.id=c15c751e94034d7591fabc8910a7076b&role.id=241e99d365564c0fa95cb712f4004785 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:35.395555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977023-a336-4859-b516-a757d9576fa3 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:35.396124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977023-a336-4859-b516-a757d9576fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:35.396124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977023-a336-4859-b516-a757d9576fa3 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:35.397369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 654/1288] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:35.402621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771148ad-2f42-47d6-9bcb-b2955f86ea8b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:35.403087 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771148ad-2f42-47d6-9bcb-b2955f86ea8b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:35.403185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771148ad-2f42-47d6-9bcb-b2955f86ea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:35.421976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-771148ad-2f42-47d6-9bcb-b2955f86ea8b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:35.423178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-771148ad-2f42-47d6-9bcb-b2955f86ea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:35.454327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:35.497729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 635/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:35.772664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977023-a336-4859-b516-a757d9576fa3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:35.883113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977023-a336-4859-b516-a757d9576fa3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:35.894019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:35.921550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:35.922169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:35.922446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:35.922739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:35.923328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:35.924711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:35.934197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:35.935368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:35.936628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 655/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:25:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:35.941405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771148ad-2f42-47d6-9bcb-b2955f86ea8b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:36.044750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771148ad-2f42-47d6-9bcb-b2955f86ea8b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:36.079840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007a0d0-2f7d-4a33-9cb5-61d6d339220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:36.080441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007a0d0-2f7d-4a33-9cb5-61d6d339220a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:36.080712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007a0d0-2f7d-4a33-9cb5-61d6d339220a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:36.080975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007a0d0-2f7d-4a33-9cb5-61d6d339220a None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:36.081605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007a0d0-2f7d-4a33-9cb5-61d6d339220a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:36.081900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007a0d0-2f7d-4a33-9cb5-61d6d339220a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:36.083301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007a0d0-2f7d-4a33-9cb5-61d6d339220a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:36.088226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 636/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Aug 31 04:25:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:36.097625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:36.212464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977d667-ae9d-4f04-8857-0f885b8a4832 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:36.250304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fa97a-73f6-43ba-87fc-bbb219b9a079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:36.250928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fa97a-73f6-43ba-87fc-bbb219b9a079 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:36.251199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fa97a-73f6-43ba-87fc-bbb219b9a079 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:36.251468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fa97a-73f6-43ba-87fc-bbb219b9a079 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:36.252093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fa97a-73f6-43ba-87fc-bbb219b9a079 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:36.252399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fa97a-73f6-43ba-87fc-bbb219b9a079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:36.253791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fa97a-73f6-43ba-87fc-bbb219b9a079 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:36.258327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 656/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Aug 31 04:25:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:38.129977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314dcb-157f-4d62-a6ea-05bdcd89db02 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:38.130830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314dcb-157f-4d62-a6ea-05bdcd89db02 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:38.130830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314dcb-157f-4d62-a6ea-05bdcd89db02 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:38.132998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 637/1293] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:38.138556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f0ad-f995-4d44-a2bc-97c9da54988f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:38.139027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f0ad-f995-4d44-a2bc-97c9da54988f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:38.139133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f0ad-f995-4d44-a2bc-97c9da54988f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:38.159542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5ad3f0ad-f995-4d44-a2bc-97c9da54988f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:38.160708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5ad3f0ad-f995-4d44-a2bc-97c9da54988f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:38.191275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:38.230027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 657/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:38.503941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7314dcb-157f-4d62-a6ea-05bdcd89db02 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:38.544352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:38.629657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7314dcb-157f-4d62-a6ea-05bdcd89db02 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:38.667621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6664053-ecc8-4567-abb8-b28fe4e97063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:38.668222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664053-ecc8-4567-abb8-b28fe4e97063 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:38.668505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664053-ecc8-4567-abb8-b28fe4e97063 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:38.668772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6664053-ecc8-4567-abb8-b28fe4e97063 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:38.669417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6664053-ecc8-4567-abb8-b28fe4e97063 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:38.670814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6664053-ecc8-4567-abb8-b28fe4e97063 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:38.689440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 638/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Aug 31 04:25:38 2022] POST /identity/v3/services => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:25:40.570305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e11194-67ce-425a-856c-1489e55d1a32 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:40.571348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e11194-67ce-425a-856c-1489e55d1a32 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:40.571348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e11194-67ce-425a-856c-1489e55d1a32 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:40.573312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 658/1296] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:40.582238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c260-ac1d-43c4-8107-c63f4ec8dbf3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:40.582675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c260-ac1d-43c4-8107-c63f4ec8dbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:40.582783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c260-ac1d-43c4-8107-c63f4ec8dbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:40.606644 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3710c260-ac1d-43c4-8107-c63f4ec8dbf3 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:40.607870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3710c260-ac1d-43c4-8107-c63f4ec8dbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:40.690479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 639/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:40.959517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e11194-67ce-425a-856c-1489e55d1a32 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:41.074990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e11194-67ce-425a-856c-1489e55d1a32 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:41.111497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:41.112126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:41.112399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:41.112662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:41.113238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:41.114622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:41.124585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:41.126848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:41.127860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 659/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:25:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:41.134128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710c260-ac1d-43c4-8107-c63f4ec8dbf3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:41.153842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:41.238195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710c260-ac1d-43c4-8107-c63f4ec8dbf3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:41.274446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:41.275073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:41.275346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:41.275721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:41.276746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:41.277085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:41.278456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:41.283776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 640/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Aug 31 04:25:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:41.290825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:41.324513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:41.404581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15372576-880b-465e-886a-ea8c311c9ff1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:41.443492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaff565-4786-4296-ba1c-b6c128a8ae7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:41.444059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaff565-4786-4296-ba1c-b6c128a8ae7a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:41.444316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaff565-4786-4296-ba1c-b6c128a8ae7a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:41.444608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaff565-4786-4296-ba1c-b6c128a8ae7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:41.445438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaff565-4786-4296-ba1c-b6c128a8ae7a None admin] RBAC: Adding query filter params (interface=public, service_id=5f944d6f577e48d8afdab2ad977aa52e, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:41.445848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaff565-4786-4296-ba1c-b6c128a8ae7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:41.447230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaff565-4786-4296-ba1c-b6c128a8ae7a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:41.452804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 660/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Aug 31 04:25:41 2022] GET /identity/v3/endpoints?service_id=5f944d6f577e48d8afdab2ad977aa52e&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:41.458743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:41.543813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:41.569010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1a8b1-1cb3-4366-8bc0-bbcde54cc5d3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:41.606175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6199d1-4c2e-459d-8db7-643dd0f807ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:41.606825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6199d1-4c2e-459d-8db7-643dd0f807ea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:41.607055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6199d1-4c2e-459d-8db7-643dd0f807ea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:41.607322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6199d1-4c2e-459d-8db7-643dd0f807ea None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:41.607917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6199d1-4c2e-459d-8db7-643dd0f807ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:41.609276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6199d1-4c2e-459d-8db7-643dd0f807ea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:41.616362 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 641/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Aug 31 04:25:41 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:25:43.472510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f1d5b-ad99-4539-b668-32440c9fdb74 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:43.473362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f1d5b-ad99-4539-b668-32440c9fdb74 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:43.473362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f1d5b-ad99-4539-b668-32440c9fdb74 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:43.475530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 661/1302] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:25:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:25:43.481672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dac15-1b16-4441-ac8b-2ae1c49be8e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:43.482357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dac15-1b16-4441-ac8b-2ae1c49be8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:43.482540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dac15-1b16-4441-ac8b-2ae1c49be8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:43.511851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b37dac15-1b16-4441-ac8b-2ae1c49be8e5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:25:43.512999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b37dac15-1b16-4441-ac8b-2ae1c49be8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:43.584767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 642/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:25:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 04:25:43.853881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f1d5b-ad99-4539-b668-32440c9fdb74 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:43.966951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:43.974526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f1d5b-ad99-4539-b668-32440c9fdb74 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:44.008392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:44.009016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:44.009286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:44.009553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:44.010153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:44.011553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 04:25:44.020505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 04:25:44.021685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 04:25:44.022747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Aug 31 04:25:44.026948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 662/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Aug 31 04:25:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 04:25:44.028322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dac15-1b16-4441-ac8b-2ae1c49be8e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:44.108729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:44.133332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dac15-1b16-4441-ac8b-2ae1c49be8e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:44.171166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002cc38-b85a-4d9b-81d8-8c13c8ac0703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:44.171800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002cc38-b85a-4d9b-81d8-8c13c8ac0703 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:44.172072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002cc38-b85a-4d9b-81d8-8c13c8ac0703 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:44.172342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002cc38-b85a-4d9b-81d8-8c13c8ac0703 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:44.172966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002cc38-b85a-4d9b-81d8-8c13c8ac0703 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:25:44.173266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002cc38-b85a-4d9b-81d8-8c13c8ac0703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:44.174645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002cc38-b85a-4d9b-81d8-8c13c8ac0703 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:44.180864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 643/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Aug 31 04:25:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:25:44.187147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:44.295056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d84562-67aa-4591-884e-3665e0b279e4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:44.329699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f023f-f4d3-4134-af6c-7f843d024901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:44.330291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f023f-f4d3-4134-af6c-7f843d024901 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:44.330566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f023f-f4d3-4134-af6c-7f843d024901 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:44.330865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f023f-f4d3-4134-af6c-7f843d024901 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:44.331522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f023f-f4d3-4134-af6c-7f843d024901 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:44.332876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f023f-f4d3-4134-af6c-7f843d024901 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:44.357882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 663/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:25:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:25:54.177177 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=30 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:54.352761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:26:48.370379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e7898-15d9-4649-9c4f-3c31d1698295 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:48.371700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e7898-15d9-4649-9c4f-3c31d1698295 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:48.372645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e7898-15d9-4649-9c4f-3c31d1698295 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:48.374609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 644/1307] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:26:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:26:48.380267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669b18-f335-4dc4-8045-4af822b535ae None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:48.381280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669b18-f335-4dc4-8045-4af822b535ae None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:48.381757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669b18-f335-4dc4-8045-4af822b535ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:48.461751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-67669b18-f335-4dc4-8045-4af822b535ae None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:26:48.463437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-67669b18-f335-4dc4-8045-4af822b535ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:48.569177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 664/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:26:48.960464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40205f0b-b91d-45e1-890a-bdacceefdbda None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:48.961303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40205f0b-b91d-45e1-890a-bdacceefdbda None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:48.961943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40205f0b-b91d-45e1-890a-bdacceefdbda None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:48.964980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 645/1309] 10.222.0.42 () {54 vars in 965 bytes} [Wed Aug 31 04:26:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:26:48.977760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f318-c344-4d72-b964-840612dc5adf None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:48.978499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f318-c344-4d72-b964-840612dc5adf None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:48.979182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f318-c344-4d72-b964-840612dc5adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:49.098885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-76a5f318-c344-4d72-b964-840612dc5adf None None] MFA Rules not processed for user `13603c6cd9544f5391256ea600ad0709`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:26:49.100913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-76a5f318-c344-4d72-b964-840612dc5adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:49.131994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:26:49.230614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 665/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Aug 31 04:26:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:26:49.232922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40205f0b-b91d-45e1-890a-bdacceefdbda None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:26:49.235681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-40205f0b-b91d-45e1-890a-bdacceefdbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:49.413548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40205f0b-b91d-45e1-890a-bdacceefdbda None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:26:49.519718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9954a-4a80-41d7-ab51-faa9eff491e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:26:49.520689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9954a-4a80-41d7-ab51-faa9eff491e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:49.521041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9954a-4a80-41d7-ab51-faa9eff491e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:49.521368 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9954a-4a80-41d7-ab51-faa9eff491e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:49.523349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-55e9954a-4a80-41d7-ab51-faa9eff491e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:49.599607 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:26:49.691373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9954a-4a80-41d7-ab51-faa9eff491e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:26:49.693620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9954a-4a80-41d7-ab51-faa9eff491e7 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:26:49.728007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 646/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:26:52.151881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004135-1c85-4d37-b80a-414d08a30df5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:52.152392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004135-1c85-4d37-b80a-414d08a30df5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:52.152506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004135-1c85-4d37-b80a-414d08a30df5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:52.153827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 666/1312] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:26:52.159400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad087e-3794-4f35-bfc8-9701f6158443 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:52.159898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad087e-3794-4f35-bfc8-9701f6158443 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:52.160073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad087e-3794-4f35-bfc8-9701f6158443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:52.272058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-88ad087e-3794-4f35-bfc8-9701f6158443 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:26:52.273438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88ad087e-3794-4f35-bfc8-9701f6158443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:52.387725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 647/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:26:52.690625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c004135-1c85-4d37-b80a-414d08a30df5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:26:52.692234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2c004135-1c85-4d37-b80a-414d08a30df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:52.799748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c004135-1c85-4d37-b80a-414d08a30df5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:26:52.877012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:26:52.879930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93693483-6779-4c9c-9de2-35b80d55e8be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:26:52.880556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93693483-6779-4c9c-9de2-35b80d55e8be service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:52.880838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93693483-6779-4c9c-9de2-35b80d55e8be service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:52.881113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93693483-6779-4c9c-9de2-35b80d55e8be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:52.883498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-93693483-6779-4c9c-9de2-35b80d55e8be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:52.995642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93693483-6779-4c9c-9de2-35b80d55e8be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:26:52.997005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93693483-6779-4c9c-9de2-35b80d55e8be service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:26:53.016359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 667/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:26:55.295931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7aba9-fffa-4e39-ac2a-f30476f65d22 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:55.296786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7aba9-fffa-4e39-ac2a-f30476f65d22 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:55.296786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7aba9-fffa-4e39-ac2a-f30476f65d22 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:55.298952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 648/1315] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:26:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:26:55.309835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ff49-54f6-4eb0-838e-ad45df968196 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:55.310290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ff49-54f6-4eb0-838e-ad45df968196 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:55.310384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83ff49-54f6-4eb0-838e-ad45df968196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:55.382088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8e83ff49-54f6-4eb0-838e-ad45df968196 None None] MFA Rules not processed for user `0d9b7166b08d4c81a876de0dd637906f`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:26:55.383258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8e83ff49-54f6-4eb0-838e-ad45df968196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:55.445768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:26:55.493722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 668/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:26:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:26:55.810927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7aba9-fffa-4e39-ac2a-f30476f65d22 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:26:55.813444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46a7aba9-fffa-4e39-ac2a-f30476f65d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:55.829217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:26:55.936010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7aba9-fffa-4e39-ac2a-f30476f65d22 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:26:56.015166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d6da2-cfed-4233-8f0a-423a1a582a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:26:56.016049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d6da2-cfed-4233-8f0a-423a1a582a2c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:56.016400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d6da2-cfed-4233-8f0a-423a1a582a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:56.017606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d6da2-cfed-4233-8f0a-423a1a582a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:56.020073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d95d6da2-cfed-4233-8f0a-423a1a582a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:56.097030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:26:56.145424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d6da2-cfed-4233-8f0a-423a1a582a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:26:56.146896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d6da2-cfed-4233-8f0a-423a1a582a2c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:26:56.165630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 649/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:26:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:26:58.627739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98edfa-6ec0-48d9-a9f6-770c5c8cc52a None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:58.628628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98edfa-6ec0-48d9-a9f6-770c5c8cc52a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:58.628628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98edfa-6ec0-48d9-a9f6-770c5c8cc52a None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:58.630649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 669/1318] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:26:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:26:58.636966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333403-b728-445d-87d1-6b5ae1fa5be7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:58.637734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333403-b728-445d-87d1-6b5ae1fa5be7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:58.637887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333403-b728-445d-87d1-6b5ae1fa5be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:58.714090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-03333403-b728-445d-87d1-6b5ae1fa5be7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:26:58.715352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-03333403-b728-445d-87d1-6b5ae1fa5be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:58.814744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 650/1319] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:26:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:26:59.098322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98edfa-6ec0-48d9-a9f6-770c5c8cc52a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:26:59.099931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7b98edfa-6ec0-48d9-a9f6-770c5c8cc52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:59.217118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98edfa-6ec0-48d9-a9f6-770c5c8cc52a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:26:59.293691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dab509-cc0d-4b05-a99f-8c60ccb5e168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:26:59.294298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab509-cc0d-4b05-a99f-8c60ccb5e168 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:26:59.294573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab509-cc0d-4b05-a99f-8c60ccb5e168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:26:59.294874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab509-cc0d-4b05-a99f-8c60ccb5e168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:26:59.296475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-08dab509-cc0d-4b05-a99f-8c60ccb5e168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:26:59.304299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:26:59.419443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dab509-cc0d-4b05-a99f-8c60ccb5e168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:26:59.420814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dab509-cc0d-4b05-a99f-8c60ccb5e168 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:26:59.442051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 670/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:26:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:01.992093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbdda6-77ea-4c1d-bcf2-b434107f8aa3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:01.992717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbdda6-77ea-4c1d-bcf2-b434107f8aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:01.992717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbdda6-77ea-4c1d-bcf2-b434107f8aa3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:01.993984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 651/1321] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:01.999241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b2ee-252d-4fdc-bac9-cde708a44f39 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:01.999664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b2ee-252d-4fdc-bac9-cde708a44f39 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:01.999795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b2ee-252d-4fdc-bac9-cde708a44f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:02.075425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5146b2ee-252d-4fdc-bac9-cde708a44f39 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:02.076658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5146b2ee-252d-4fdc-bac9-cde708a44f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:02.138197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:02.180500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 671/1322] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:02.460411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbdda6-77ea-4c1d-bcf2-b434107f8aa3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:02.461985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c6fbdda6-77ea-4c1d-bcf2-b434107f8aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:02.482893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:02.570490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbdda6-77ea-4c1d-bcf2-b434107f8aa3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:02.647030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7551e-e8ba-4bb2-94e7-6caba19d65a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:02.647521 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7551e-e8ba-4bb2-94e7-6caba19d65a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:02.647798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7551e-e8ba-4bb2-94e7-6caba19d65a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:02.648061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7551e-e8ba-4bb2-94e7-6caba19d65a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:02.649699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7bb7551e-e8ba-4bb2-94e7-6caba19d65a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:02.728329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:02.767449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7551e-e8ba-4bb2-94e7-6caba19d65a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:02.768889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7551e-e8ba-4bb2-94e7-6caba19d65a4 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:02.788184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 652/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:04.750278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc5c53-a89a-4697-8f9c-706c04a64d1a None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:04.750825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc5c53-a89a-4697-8f9c-706c04a64d1a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:04.750825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc5c53-a89a-4697-8f9c-706c04a64d1a None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:04.752095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 672/1324] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:04.757399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ae40-7b20-4753-8e4c-8d43519a18a0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:04.757878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ae40-7b20-4753-8e4c-8d43519a18a0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:04.757878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9ae40-7b20-4753-8e4c-8d43519a18a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:04.827495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-78f9ae40-7b20-4753-8e4c-8d43519a18a0 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:04.828741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-78f9ae40-7b20-4753-8e4c-8d43519a18a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:04.931746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 653/1325] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:05.213935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc5c53-a89a-4697-8f9c-706c04a64d1a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:05.215628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7dcc5c53-a89a-4697-8f9c-706c04a64d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:05.330559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc5c53-a89a-4697-8f9c-706c04a64d1a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:05.405653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b9960-fe73-4aa0-98dd-78d77e9320dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:05.406286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9960-fe73-4aa0-98dd-78d77e9320dc service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:05.406559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9960-fe73-4aa0-98dd-78d77e9320dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:05.406859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9960-fe73-4aa0-98dd-78d77e9320dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:05.408618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-539b9960-fe73-4aa0-98dd-78d77e9320dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:05.425663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:05.521745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b9960-fe73-4aa0-98dd-78d77e9320dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:05.523155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b9960-fe73-4aa0-98dd-78d77e9320dc service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:05.541086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 673/1326] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:07.481521 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c887-24d4-4f7c-ae21-5e66503c1ff4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:07.482147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c887-24d4-4f7c-ae21-5e66503c1ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:07.482147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c887-24d4-4f7c-ae21-5e66503c1ff4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:07.483467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 654/1327] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:07.489490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7318f21-7461-4492-b5b6-cbeb24ff8ab7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:07.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7318f21-7461-4492-b5b6-cbeb24ff8ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:07.490046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7318f21-7461-4492-b5b6-cbeb24ff8ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:07.558816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e7318f21-7461-4492-b5b6-cbeb24ff8ab7 None None] MFA Rules not processed for user `0d9b7166b08d4c81a876de0dd637906f`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:07.559977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e7318f21-7461-4492-b5b6-cbeb24ff8ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:07.626569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:07.661334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 674/1328] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:07.940617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896c887-24d4-4f7c-ae21-5e66503c1ff4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:07.942161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2896c887-24d4-4f7c-ae21-5e66503c1ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:07.978693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:08.052916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896c887-24d4-4f7c-ae21-5e66503c1ff4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:08.130825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafaaf1-df99-4ae5-b168-4700955bc315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:08.131363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafaaf1-df99-4ae5-b168-4700955bc315 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:08.131636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafaaf1-df99-4ae5-b168-4700955bc315 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:08.131898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafaaf1-df99-4ae5-b168-4700955bc315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:08.133498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1cafaaf1-df99-4ae5-b168-4700955bc315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:08.215490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:08.248767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafaaf1-df99-4ae5-b168-4700955bc315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:08.250244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafaaf1-df99-4ae5-b168-4700955bc315 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:08.269489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 655/1329] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:10.411532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370f9b-2ebe-4804-98ec-b1e6e8ca3e11 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:10.412077 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370f9b-2ebe-4804-98ec-b1e6e8ca3e11 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:10.412077 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370f9b-2ebe-4804-98ec-b1e6e8ca3e11 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:10.413430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 675/1330] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:10.418690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ba40-2162-401e-8ff2-0a5103d9c418 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:10.419137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ba40-2162-401e-8ff2-0a5103d9c418 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:10.419282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ba40-2162-401e-8ff2-0a5103d9c418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:10.492850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fe25ba40-2162-401e-8ff2-0a5103d9c418 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:10.494094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe25ba40-2162-401e-8ff2-0a5103d9c418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:10.597465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 656/1331] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:10.898555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecdd91-f084-4eb6-94e7-6017039439d8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:10.898845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecdd91-f084-4eb6-94e7-6017039439d8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:10.899163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecdd91-f084-4eb6-94e7-6017039439d8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:10.901183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 676/1332] 10.222.0.42 () {54 vars in 965 bytes} [Wed Aug 31 04:27:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:10.907359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a6dd1-7af0-4cd6-bde6-447ec19baba3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:10.907669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a6dd1-7af0-4cd6-bde6-447ec19baba3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:10.907948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a6dd1-7af0-4cd6-bde6-447ec19baba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:10.955709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:10.989397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c10a6dd1-7af0-4cd6-bde6-447ec19baba3 None None] MFA Rules not processed for user `13603c6cd9544f5391256ea600ad0709`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:10.990608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c10a6dd1-7af0-4cd6-bde6-447ec19baba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:11.071425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 657/1333] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Aug 31 04:27:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:27:11.081062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecdd91-f084-4eb6-94e7-6017039439d8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:11.082588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-55ecdd91-f084-4eb6-94e7-6017039439d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:11.194460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecdd91-f084-4eb6-94e7-6017039439d8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:11.276909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e0528-834e-47e3-bfbd-162cda21dd8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:11.277573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e0528-834e-47e3-bfbd-162cda21dd8d service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:11.277828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e0528-834e-47e3-bfbd-162cda21dd8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:11.278059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e0528-834e-47e3-bfbd-162cda21dd8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:11.279709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-143e0528-834e-47e3-bfbd-162cda21dd8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:11.308057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:11.413463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e0528-834e-47e3-bfbd-162cda21dd8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:11.414946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e0528-834e-47e3-bfbd-162cda21dd8d service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:11.435171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 677/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:13.579171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb385e-7b6e-4962-8d83-a2ea39c7f2e9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:13.580121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb385e-7b6e-4962-8d83-a2ea39c7f2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:13.580121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb385e-7b6e-4962-8d83-a2ea39c7f2e9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:13.582285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 658/1335] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:13.587653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d640f-789f-4bf1-a2c3-05458377f2bb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:13.588095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d640f-789f-4bf1-a2c3-05458377f2bb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:13.588204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d640f-789f-4bf1-a2c3-05458377f2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:13.654958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-423d640f-789f-4bf1-a2c3-05458377f2bb None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:13.656144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-423d640f-789f-4bf1-a2c3-05458377f2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:13.735208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:13.760328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 678/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:14.045608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb385e-7b6e-4962-8d83-a2ea39c7f2e9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:14.047198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-39eb385e-7b6e-4962-8d83-a2ea39c7f2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:14.158646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb385e-7b6e-4962-8d83-a2ea39c7f2e9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:14.187038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:14.234073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8fbd4-4216-4bd5-a696-abe187901215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:14.234664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8fbd4-4216-4bd5-a696-abe187901215 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:14.234971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8fbd4-4216-4bd5-a696-abe187901215 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:14.235244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8fbd4-4216-4bd5-a696-abe187901215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:14.236835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88f8fbd4-4216-4bd5-a696-abe187901215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:14.354603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8fbd4-4216-4bd5-a696-abe187901215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:14.356069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8fbd4-4216-4bd5-a696-abe187901215 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:14.374373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 659/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:16.900940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1db4b-9317-44ec-9464-31efeb5e24e9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:16.901498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1db4b-9317-44ec-9464-31efeb5e24e9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:16.901615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1db4b-9317-44ec-9464-31efeb5e24e9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:16.903593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 679/1338] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:16.909041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044227ab-2ba3-4221-a3f8-554308eba15f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:16.909511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044227ab-2ba3-4221-a3f8-554308eba15f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:16.909625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044227ab-2ba3-4221-a3f8-554308eba15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:16.976619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-044227ab-2ba3-4221-a3f8-554308eba15f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:16.977815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-044227ab-2ba3-4221-a3f8-554308eba15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:16.986683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:17.086342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 660/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:17.366222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1db4b-9317-44ec-9464-31efeb5e24e9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:17.367807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-60b1db4b-9317-44ec-9464-31efeb5e24e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:17.491995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1db4b-9317-44ec-9464-31efeb5e24e9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:17.576051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1053b-8635-40ec-b89d-779df6937f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:17.576868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1053b-8635-40ec-b89d-779df6937f55 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:17.577217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1053b-8635-40ec-b89d-779df6937f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:17.577872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1053b-8635-40ec-b89d-779df6937f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:17.580180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-16f1053b-8635-40ec-b89d-779df6937f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:17.619215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:17.710737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1053b-8635-40ec-b89d-779df6937f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:17.712180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1053b-8635-40ec-b89d-779df6937f55 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:17.731384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 680/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:21.738235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a226af-6825-48af-aa83-96565f1e90a4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:21.738798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a226af-6825-48af-aa83-96565f1e90a4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:21.738798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a226af-6825-48af-aa83-96565f1e90a4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:21.740158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 661/1341] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:21.745416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4d00f-ecda-4906-b04b-6c36206194a9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:21.745840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4d00f-ecda-4906-b04b-6c36206194a9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:21.745993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4d00f-ecda-4906-b04b-6c36206194a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:21.813037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6cf4d00f-ecda-4906-b04b-6c36206194a9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:21.814301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6cf4d00f-ecda-4906-b04b-6c36206194a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:21.901386 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:21.913576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 681/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:22.187587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a226af-6825-48af-aa83-96565f1e90a4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:22.190039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d1a226af-6825-48af-aa83-96565f1e90a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:22.304013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a226af-6825-48af-aa83-96565f1e90a4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:22.343234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:22.382262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d288-9bb0-44dd-bfda-a28942262a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:22.382909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d288-9bb0-44dd-bfda-a28942262a2a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:22.383190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d288-9bb0-44dd-bfda-a28942262a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:22.383462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d288-9bb0-44dd-bfda-a28942262a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:22.385088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9d26d288-9bb0-44dd-bfda-a28942262a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:22.503511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d288-9bb0-44dd-bfda-a28942262a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:22.504853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d288-9bb0-44dd-bfda-a28942262a2a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:22.524196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 662/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:25.173331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51989a4-6e61-4165-8276-7d5ebf7699b2 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:25.173959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51989a4-6e61-4165-8276-7d5ebf7699b2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:25.174063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51989a4-6e61-4165-8276-7d5ebf7699b2 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:25.175486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 682/1344] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:25.181508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad36c-319a-4088-ad64-b598f0be14ff None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:25.181983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad36c-319a-4088-ad64-b598f0be14ff None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:25.182118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad36c-319a-4088-ad64-b598f0be14ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:25.261662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7b2ad36c-319a-4088-ad64-b598f0be14ff None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:25.262949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7b2ad36c-319a-4088-ad64-b598f0be14ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:25.283415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:25.394421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 663/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:25.679481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51989a4-6e61-4165-8276-7d5ebf7699b2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:25.681026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d51989a4-6e61-4165-8276-7d5ebf7699b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:25.797160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51989a4-6e61-4165-8276-7d5ebf7699b2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:25.872438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96c20d-d9e4-47e8-9fd9-200608262fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:25.873053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96c20d-d9e4-47e8-9fd9-200608262fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:25.873332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96c20d-d9e4-47e8-9fd9-200608262fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:25.873601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96c20d-d9e4-47e8-9fd9-200608262fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:25.875236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0c96c20d-d9e4-47e8-9fd9-200608262fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:25.919831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:25.992422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96c20d-d9e4-47e8-9fd9-200608262fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:25.993893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96c20d-d9e4-47e8-9fd9-200608262fc2 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:26.012909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 683/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:29.851706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c783d-38f7-4a2d-82c5-fc597f96da3c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:29.852276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c783d-38f7-4a2d-82c5-fc597f96da3c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:29.852276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c783d-38f7-4a2d-82c5-fc597f96da3c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:29.853593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 664/1347] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:29.859383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62185f2d-03ef-4cb2-8c62-bee02713321c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:29.859916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62185f2d-03ef-4cb2-8c62-bee02713321c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:29.859916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62185f2d-03ef-4cb2-8c62-bee02713321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:29.931131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-62185f2d-03ef-4cb2-8c62-bee02713321c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:29.932362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-62185f2d-03ef-4cb2-8c62-bee02713321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:30.032276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:30.035626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 684/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:30.320046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c783d-38f7-4a2d-82c5-fc597f96da3c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:30.322817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df9c783d-38f7-4a2d-82c5-fc597f96da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:30.441339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c783d-38f7-4a2d-82c5-fc597f96da3c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:30.491084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:30.517689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46708802-4506-4f09-af13-a3a22bb4e2b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:30.518401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46708802-4506-4f09-af13-a3a22bb4e2b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:30.518675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46708802-4506-4f09-af13-a3a22bb4e2b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:30.518997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46708802-4506-4f09-af13-a3a22bb4e2b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:30.520585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46708802-4506-4f09-af13-a3a22bb4e2b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:30.634209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46708802-4506-4f09-af13-a3a22bb4e2b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:30.635669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46708802-4506-4f09-af13-a3a22bb4e2b8 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:30.654755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 665/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:35.539264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869ed0a-9151-4c96-8255-51a370326b2e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:35.539738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869ed0a-9151-4c96-8255-51a370326b2e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:35.540235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869ed0a-9151-4c96-8255-51a370326b2e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:35.541562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 685/1350] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:35.546614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a969e-5940-43c4-a239-f13689a2b8aa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:35.548031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a969e-5940-43c4-a239-f13689a2b8aa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:35.548031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a969e-5940-43c4-a239-f13689a2b8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:35.681677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3e3a969e-5940-43c4-a239-f13689a2b8aa None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:35.683124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e3a969e-5940-43c4-a239-f13689a2b8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:35.739198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:35.844644 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 666/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:27:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:39.863588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8378a-0735-4f30-8d44-3aaa3d47317f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:39.864168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8378a-0735-4f30-8d44-3aaa3d47317f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:39.864168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8378a-0735-4f30-8d44-3aaa3d47317f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:39.865667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 686/1352] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:39.870896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a196a-c159-47be-81ed-f6c4a99a4aba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:39.871647 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a196a-c159-47be-81ed-f6c4a99a4aba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:39.872067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a196a-c159-47be-81ed-f6c4a99a4aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:39.960813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6f1a196a-c159-47be-81ed-f6c4a99a4aba None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:39.962354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6f1a196a-c159-47be-81ed-f6c4a99a4aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:40.046283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:40.098339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 667/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:27:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:41.593116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcb0d8-d7ee-4540-a785-7e99be0c5d2e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:41.593690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcb0d8-d7ee-4540-a785-7e99be0c5d2e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:41.593690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcb0d8-d7ee-4540-a785-7e99be0c5d2e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:41.595216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 687/1354] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:41.600593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc15ed-b621-4edb-9192-547419ea09ba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:41.601044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc15ed-b621-4edb-9192-547419ea09ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:41.601266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc15ed-b621-4edb-9192-547419ea09ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:41.682951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-89dc15ed-b621-4edb-9192-547419ea09ba None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:41.684164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-89dc15ed-b621-4edb-9192-547419ea09ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:41.800367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 668/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:27:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:27:45.027988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9abe74-f77f-421a-90b0-3fa7dd8104ce None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:45.028557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9abe74-f77f-421a-90b0-3fa7dd8104ce None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:45.028557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9abe74-f77f-421a-90b0-3fa7dd8104ce None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:45.030758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 688/1356] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:45.036387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8f47d-23ca-4783-bb1b-059525f54adf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:45.036820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8f47d-23ca-4783-bb1b-059525f54adf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:45.036910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8f47d-23ca-4783-bb1b-059525f54adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:45.079823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:45.120383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-39c8f47d-23ca-4783-bb1b-059525f54adf None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:45.121651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-39c8f47d-23ca-4783-bb1b-059525f54adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:45.238880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 669/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:45.542147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c4ec0-795f-4bf8-865b-a580a685e2ef None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:45.542596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c4ec0-795f-4bf8-865b-a580a685e2ef None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:45.543056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c4ec0-795f-4bf8-865b-a580a685e2ef None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:45.544951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 689/1358] 10.222.0.42 () {54 vars in 962 bytes} [Wed Aug 31 04:27:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:45.553748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44250-bbdc-4fba-95ff-b0260b1d3f3b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:45.555484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44250-bbdc-4fba-95ff-b0260b1d3f3b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:45.557044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44250-bbdc-4fba-95ff-b0260b1d3f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:45.639236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-38d44250-bbdc-4fba-95ff-b0260b1d3f3b None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:45.640432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38d44250-bbdc-4fba-95ff-b0260b1d3f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:45.704684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:45.775757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 670/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Aug 31 04:27:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:45.783407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c4ec0-795f-4bf8-865b-a580a685e2ef None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:45.785156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4e3c4ec0-795f-4bf8-865b-a580a685e2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:45.936786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c4ec0-795f-4bf8-865b-a580a685e2ef None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:46.013359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:46.057433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df76e7-c9ba-4680-8450-fa90688176b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:46.058104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df76e7-c9ba-4680-8450-fa90688176b3 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:46.058360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df76e7-c9ba-4680-8450-fa90688176b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:46.058838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df76e7-c9ba-4680-8450-fa90688176b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:46.060780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-96df76e7-c9ba-4680-8450-fa90688176b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:46.158442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df76e7-c9ba-4680-8450-fa90688176b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:46.160196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df76e7-c9ba-4680-8450-fa90688176b3 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:46.179065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 690/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:48.295470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eac50-1a63-4dd0-b72e-a719b5c789fb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:48.295874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eac50-1a63-4dd0-b72e-a719b5c789fb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:48.296044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eac50-1a63-4dd0-b72e-a719b5c789fb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:48.297517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 671/1361] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:48.304976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b27e4-bb19-424a-bb62-ecdf39a30772 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:48.305411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b27e4-bb19-424a-bb62-ecdf39a30772 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:48.305491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b27e4-bb19-424a-bb62-ecdf39a30772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:48.380840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-da3b27e4-bb19-424a-bb62-ecdf39a30772 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:48.382231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-da3b27e4-bb19-424a-bb62-ecdf39a30772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:48.428423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:48.496030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 691/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:48.796489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eac50-1a63-4dd0-b72e-a719b5c789fb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:48.797669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-931eac50-1a63-4dd0-b72e-a719b5c789fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:49.005120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eac50-1a63-4dd0-b72e-a719b5c789fb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:49.015577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:49.130930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eeb2da-17c4-4a55-aa04-c321751e1b48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:49.131636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eeb2da-17c4-4a55-aa04-c321751e1b48 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:49.131918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eeb2da-17c4-4a55-aa04-c321751e1b48 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:49.132183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eeb2da-17c4-4a55-aa04-c321751e1b48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:49.133782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c6eeb2da-17c4-4a55-aa04-c321751e1b48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:49.240943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eeb2da-17c4-4a55-aa04-c321751e1b48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:49.242784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eeb2da-17c4-4a55-aa04-c321751e1b48 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:49.269411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 672/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:51.480728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66602dc3-16a5-49a4-abde-7cf3427f7b25 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:51.481290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66602dc3-16a5-49a4-abde-7cf3427f7b25 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:51.481290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66602dc3-16a5-49a4-abde-7cf3427f7b25 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:51.482543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 692/1364] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:51.487937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e705a-9425-41fb-a39f-44a5aaf5b18a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:51.488358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e705a-9425-41fb-a39f-44a5aaf5b18a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:51.488528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e705a-9425-41fb-a39f-44a5aaf5b18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:51.504982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:51.560405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-807e705a-9425-41fb-a39f-44a5aaf5b18a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:51.561595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-807e705a-9425-41fb-a39f-44a5aaf5b18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:51.666826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 673/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:51.965815 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf04a54-3ca5-4189-94e0-16dbfa14bd8c None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:51.966430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf04a54-3ca5-4189-94e0-16dbfa14bd8c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:51.966821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf04a54-3ca5-4189-94e0-16dbfa14bd8c None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:51.968295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 693/1366] 10.222.0.42 () {54 vars in 962 bytes} [Wed Aug 31 04:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:51.974604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b270bd-1725-4663-8f0c-22fa098c3afe None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:51.975045 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b270bd-1725-4663-8f0c-22fa098c3afe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:51.975409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b270bd-1725-4663-8f0c-22fa098c3afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:52.071866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-96b270bd-1725-4663-8f0c-22fa098c3afe None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:52.073278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-96b270bd-1725-4663-8f0c-22fa098c3afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:52.112358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:52.201989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 674/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Aug 31 04:27:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:52.210634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf04a54-3ca5-4189-94e0-16dbfa14bd8c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:52.212233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-baf04a54-3ca5-4189-94e0-16dbfa14bd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:52.377130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf04a54-3ca5-4189-94e0-16dbfa14bd8c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:52.401134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:52.504143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa7677-ccd6-4b9f-9c40-e81b96b2d579 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:52.504806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa7677-ccd6-4b9f-9c40-e81b96b2d579 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:52.505066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa7677-ccd6-4b9f-9c40-e81b96b2d579 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:52.505169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa7677-ccd6-4b9f-9c40-e81b96b2d579 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:52.506822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f2fa7677-ccd6-4b9f-9c40-e81b96b2d579 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:52.600368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa7677-ccd6-4b9f-9c40-e81b96b2d579 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:52.601993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa7677-ccd6-4b9f-9c40-e81b96b2d579 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:52.624032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 694/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:53.190055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfbc07-f726-41bf-948d-266aeb07a965 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:53.191043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfbc07-f726-41bf-948d-266aeb07a965 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:53.191043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfbc07-f726-41bf-948d-266aeb07a965 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:53.192424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 675/1369] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:53.197518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729a63b-b226-4d96-a8d6-70152e22daf2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:53.197806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729a63b-b226-4d96-a8d6-70152e22daf2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:53.198082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729a63b-b226-4d96-a8d6-70152e22daf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:53.231290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:53.267525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9729a63b-b226-4d96-a8d6-70152e22daf2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:53.268825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9729a63b-b226-4d96-a8d6-70152e22daf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:53.370585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 695/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:53.947346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52432-2d87-4ffa-8802-616b9b07366f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:53.947346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52432-2d87-4ffa-8802-616b9b07366f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:53.947346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a52432-2d87-4ffa-8802-616b9b07366f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:53.950074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 676/1371] 10.222.0.42 () {54 vars in 964 bytes} [Wed Aug 31 04:27:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:53.952920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c98b-a9e7-49b9-931b-db0a99a76d9f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:53.953465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c98b-a9e7-49b9-931b-db0a99a76d9f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:53.953465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c98b-a9e7-49b9-931b-db0a99a76d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:54.034643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b6d1c98b-a9e7-49b9-931b-db0a99a76d9f None None] MFA Rules not processed for user `6f3a86b76cff4b48a382888cbac913f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:54.036087 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b6d1c98b-a9e7-49b9-931b-db0a99a76d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:54.085703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:54.122438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 696/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Aug 31 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:54.129861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a52432-2d87-4ffa-8802-616b9b07366f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:54.131565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e3a52432-2d87-4ffa-8802-616b9b07366f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:54.242517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a52432-2d87-4ffa-8802-616b9b07366f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:54.283076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:54.320695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde6ef8-5f55-49e7-92d9-6cda37267490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:54.321426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde6ef8-5f55-49e7-92d9-6cda37267490 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:54.321872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde6ef8-5f55-49e7-92d9-6cda37267490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:54.322257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde6ef8-5f55-49e7-92d9-6cda37267490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:54.323941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5dde6ef8-5f55-49e7-92d9-6cda37267490 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:54.452074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde6ef8-5f55-49e7-92d9-6cda37267490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:54.453517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde6ef8-5f55-49e7-92d9-6cda37267490 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:54.475612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 677/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:54.596102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d081d9-41f6-4bfe-bfbf-d5e0c43920e8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:54.596460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d081d9-41f6-4bfe-bfbf-d5e0c43920e8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:54.596743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d081d9-41f6-4bfe-bfbf-d5e0c43920e8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:54.598109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 697/1374] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:54.604529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06be05-74f0-4335-8879-814c137a0ddb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:54.605038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06be05-74f0-4335-8879-814c137a0ddb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:54.605461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06be05-74f0-4335-8879-814c137a0ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:54.701285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9a06be05-74f0-4335-8879-814c137a0ddb None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:54.702978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a06be05-74f0-4335-8879-814c137a0ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:54.726468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:54.809575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 678/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:55.097195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d081d9-41f6-4bfe-bfbf-d5e0c43920e8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:55.098777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e3d081d9-41f6-4bfe-bfbf-d5e0c43920e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:55.252823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d081d9-41f6-4bfe-bfbf-d5e0c43920e8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:55.308834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:55.373264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9585a-9e7a-4614-9c7c-d96a860053ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:55.373781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9585a-9e7a-4614-9c7c-d96a860053ba service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:55.374057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9585a-9e7a-4614-9c7c-d96a860053ba service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:55.374364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9585a-9e7a-4614-9c7c-d96a860053ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:55.376056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c1a9585a-9e7a-4614-9c7c-d96a860053ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:55.472582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9585a-9e7a-4614-9c7c-d96a860053ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:55.474352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9585a-9e7a-4614-9c7c-d96a860053ba service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:55.492855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 698/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:27:57.433315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667250f-6edc-457b-8295-26ad0e442afb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:57.433870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667250f-6edc-457b-8295-26ad0e442afb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:57.433870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667250f-6edc-457b-8295-26ad0e442afb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:57.435233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 679/1377] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:27:57.440582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0492-a1b5-4db6-bd83-80050ee60de2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:57.441015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0492-a1b5-4db6-bd83-80050ee60de2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:57.441109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd0492-a1b5-4db6-bd83-80050ee60de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:57.513893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-46bd0492-a1b5-4db6-bd83-80050ee60de2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:27:57.515151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-46bd0492-a1b5-4db6-bd83-80050ee60de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:57.533266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:57.634531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 699/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:27:57.927457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667250f-6edc-457b-8295-26ad0e442afb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:27:57.929084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f667250f-6edc-457b-8295-26ad0e442afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:58.137930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667250f-6edc-457b-8295-26ad0e442afb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:27:58.153024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:27:58.258846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da660538-79c0-42b1-87b7-61602a3394cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:27:58.258846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660538-79c0-42b1-87b7-61602a3394cf service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:27:58.258846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660538-79c0-42b1-87b7-61602a3394cf service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:27:58.258846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660538-79c0-42b1-87b7-61602a3394cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:27:58.259910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-da660538-79c0-42b1-87b7-61602a3394cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:27:58.352814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da660538-79c0-42b1-87b7-61602a3394cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:27:58.354414 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da660538-79c0-42b1-87b7-61602a3394cf service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:27:58.374463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 680/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:00.300556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5300-f71f-4635-860b-3f398cc7560d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:00.301095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5300-f71f-4635-860b-3f398cc7560d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:00.301095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5300-f71f-4635-860b-3f398cc7560d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:00.303231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 700/1380] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:00.307821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76287-36ec-4c81-b5d4-072161418541 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:00.308270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76287-36ec-4c81-b5d4-072161418541 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:00.308440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c76287-36ec-4c81-b5d4-072161418541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:00.320671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:00.384179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-93c76287-36ec-4c81-b5d4-072161418541 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:00.385367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-93c76287-36ec-4c81-b5d4-072161418541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:00.494768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 681/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:00.773834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed5300-f71f-4635-860b-3f398cc7560d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:00.775377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c6ed5300-f71f-4635-860b-3f398cc7560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:00.926469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:00.937235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed5300-f71f-4635-860b-3f398cc7560d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:01.055410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0fee3-6189-4904-8560-12b6d1fd27c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:01.056045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0fee3-6189-4904-8560-12b6d1fd27c6 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:01.056325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0fee3-6189-4904-8560-12b6d1fd27c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:01.056598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0fee3-6189-4904-8560-12b6d1fd27c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:01.058235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-38d0fee3-6189-4904-8560-12b6d1fd27c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:01.166816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0fee3-6189-4904-8560-12b6d1fd27c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:01.168597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0fee3-6189-4904-8560-12b6d1fd27c6 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:01.180388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:01.190308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 701/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:03.124656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c193d-25e4-47a6-80e5-438f94733faa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:03.125267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c193d-25e4-47a6-80e5-438f94733faa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:03.125267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c193d-25e4-47a6-80e5-438f94733faa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:03.126652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 682/1383] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:03.132758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329d861-d7c5-426d-b63f-cea209cedc5d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:03.133355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329d861-d7c5-426d-b63f-cea209cedc5d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:03.133355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329d861-d7c5-426d-b63f-cea209cedc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:03.226455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8329d861-d7c5-426d-b63f-cea209cedc5d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:03.228086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8329d861-d7c5-426d-b63f-cea209cedc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:03.352901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 702/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:28:03.763403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c193d-25e4-47a6-80e5-438f94733faa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:03.765006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-985c193d-25e4-47a6-80e5-438f94733faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:03.861519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:03.931863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c193d-25e4-47a6-80e5-438f94733faa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:04.086826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eac71-21ee-4569-a110-e59d88668b09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:04.087485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eac71-21ee-4569-a110-e59d88668b09 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:04.087781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eac71-21ee-4569-a110-e59d88668b09 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:04.088057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eac71-21ee-4569-a110-e59d88668b09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:04.089969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe7eac71-21ee-4569-a110-e59d88668b09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:04.147851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:04.193039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7eac71-21ee-4569-a110-e59d88668b09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:04.195499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7eac71-21ee-4569-a110-e59d88668b09 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:04.222923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 683/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:06.158384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931adfa-8038-43a3-bced-6b9af07fab54 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:06.158997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931adfa-8038-43a3-bced-6b9af07fab54 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:06.158997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931adfa-8038-43a3-bced-6b9af07fab54 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:06.160439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 703/1386] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:06.166476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bae99e-3a54-4f5e-b8f4-e049c4a715af None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:06.167004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bae99e-3a54-4f5e-b8f4-e049c4a715af None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:06.167250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bae99e-3a54-4f5e-b8f4-e049c4a715af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:06.240276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-63bae99e-3a54-4f5e-b8f4-e049c4a715af None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:06.241432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-63bae99e-3a54-4f5e-b8f4-e049c4a715af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:06.341537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 684/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:06.647775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931adfa-8038-43a3-bced-6b9af07fab54 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:06.649521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a931adfa-8038-43a3-bced-6b9af07fab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:06.750086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:06.825349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931adfa-8038-43a3-bced-6b9af07fab54 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:06.955506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30bc19-2a23-464d-aa43-39da27cda9d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:06.956526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30bc19-2a23-464d-aa43-39da27cda9d5 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:06.956985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30bc19-2a23-464d-aa43-39da27cda9d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:06.957264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30bc19-2a23-464d-aa43-39da27cda9d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:06.958997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fa30bc19-2a23-464d-aa43-39da27cda9d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:07.003084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:07.067867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30bc19-2a23-464d-aa43-39da27cda9d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:07.069560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30bc19-2a23-464d-aa43-39da27cda9d5 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:07.089830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 704/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:09.046991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba681-d6da-469d-b2dc-26749d0b3912 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:09.047606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba681-d6da-469d-b2dc-26749d0b3912 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:09.047606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba681-d6da-469d-b2dc-26749d0b3912 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:09.048939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 685/1389] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:09.054910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd22b6-572e-49d5-9bc0-fb3427ed7040 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:09.055392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd22b6-572e-49d5-9bc0-fb3427ed7040 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:09.055606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd22b6-572e-49d5-9bc0-fb3427ed7040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:09.125337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4fbd22b6-572e-49d5-9bc0-fb3427ed7040 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:09.126563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4fbd22b6-572e-49d5-9bc0-fb3427ed7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:09.232567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 705/1390] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:09.445790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766fcc7-d068-49a4-b3f0-564b998addea None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:09.446129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766fcc7-d068-49a4-b3f0-564b998addea None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:09.446375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766fcc7-d068-49a4-b3f0-564b998addea None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:09.447775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 686/1391] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:09.453169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33dfcf-bc79-4c0e-aa2d-c0774f533c37 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:09.453453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33dfcf-bc79-4c0e-aa2d-c0774f533c37 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:09.453722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33dfcf-bc79-4c0e-aa2d-c0774f533c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:09.470842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:09.515858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766fcc7-d068-49a4-b3f0-564b998addea None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:09.517428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6766fcc7-d068-49a4-b3f0-564b998addea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:09.522676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fa33dfcf-bc79-4c0e-aa2d-c0774f533c37 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:09.523886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fa33dfcf-bc79-4c0e-aa2d-c0774f533c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:09.548800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:09.623442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 706/1392] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:09.665687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766fcc7-d068-49a4-b3f0-564b998addea None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:09.773990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:09.786062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cb3cf-0df5-4e9d-9824-477c0984ac37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:09.786674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb3cf-0df5-4e9d-9824-477c0984ac37 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:09.786973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb3cf-0df5-4e9d-9824-477c0984ac37 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:09.787237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb3cf-0df5-4e9d-9824-477c0984ac37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:09.788814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a06cb3cf-0df5-4e9d-9824-477c0984ac37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:09.879289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cb3cf-0df5-4e9d-9824-477c0984ac37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:09.880907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cb3cf-0df5-4e9d-9824-477c0984ac37 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:09.899465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 687/1393] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:09.921135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdbeb-cd23-4b2a-a29d-7883d90bf5bd None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:09.921418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdbeb-cd23-4b2a-a29d-7883d90bf5bd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:09.921687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdbeb-cd23-4b2a-a29d-7883d90bf5bd None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:09.923103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 707/1394] 10.222.0.42 () {54 vars in 967 bytes} [Wed Aug 31 04:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:09.928640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4ef2-1ebd-4626-9040-db3c71ed5edb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:09.928936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4ef2-1ebd-4626-9040-db3c71ed5edb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:09.929213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4ef2-1ebd-4626-9040-db3c71ed5edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:10.015039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-472d4ef2-1ebd-4626-9040-db3c71ed5edb None None] MFA Rules not processed for user `ca15f81281dc42f4befeb9335698d18f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:10.016291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-472d4ef2-1ebd-4626-9040-db3c71ed5edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:10.078769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:10.105942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 688/1395] 10.222.0.42 () {58 vars in 1064 bytes} [Wed Aug 31 04:28:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:10.114138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffdbeb-cd23-4b2a-a29d-7883d90bf5bd None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:10.115721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cbffdbeb-cd23-4b2a-a29d-7883d90bf5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:10.215129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:10.223856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffdbeb-cd23-4b2a-a29d-7883d90bf5bd None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:10.299971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76d271-a6f4-4be3-ac97-9516902b130d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:10.300663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d271-a6f4-4be3-ac97-9516902b130d service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:10.300943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d271-a6f4-4be3-ac97-9516902b130d service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:10.301213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d271-a6f4-4be3-ac97-9516902b130d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:10.302834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7e76d271-a6f4-4be3-ac97-9516902b130d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:10.415748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76d271-a6f4-4be3-ac97-9516902b130d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:10.417124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76d271-a6f4-4be3-ac97-9516902b130d service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:10.436368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 708/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:11.178191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8623-384d-486f-9805-789bd7977721 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:11.178672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8623-384d-486f-9805-789bd7977721 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:11.178803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8623-384d-486f-9805-789bd7977721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:11.247222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a82a8623-384d-486f-9805-789bd7977721 None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:11.248411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a82a8623-384d-486f-9805-789bd7977721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:11.345690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 689/1397] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:28:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:11.427128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e51c6-df27-4375-917b-f649fbd3f277 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:11.428083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e51c6-df27-4375-917b-f649fbd3f277 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:11.428447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e51c6-df27-4375-917b-f649fbd3f277 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:11.430154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 709/1398] 10.222.0.42 () {54 vars in 943 bytes} [Wed Aug 31 04:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:11.436969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0b22e-b263-4271-ad4c-93d6d086a7f5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:11.437438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0b22e-b263-4271-ad4c-93d6d086a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:11.437760 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0b22e-b263-4271-ad4c-93d6d086a7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:11.540038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fdf0b22e-b263-4271-ad4c-93d6d086a7f5 None None] MFA Rules not processed for user `5e4604fb28234efc9f107cb1f4aa57ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:11.541451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fdf0b22e-b263-4271-ad4c-93d6d086a7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:11.557889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:11.660902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 690/1399] 10.222.0.42 () {58 vars in 1040 bytes} [Wed Aug 31 04:28:11 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:28:11.670181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e51c6-df27-4375-917b-f649fbd3f277 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:11.671749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3d4e51c6-df27-4375-917b-f649fbd3f277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:11.700288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:11.820380 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e51c6-df27-4375-917b-f649fbd3f277 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:11.841135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77658efd-da49-4780-9397-a35bea764739 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:11.841442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77658efd-da49-4780-9397-a35bea764739 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:11.841769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77658efd-da49-4780-9397-a35bea764739 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:11.843852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 691/1400] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:11.849983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff40b60-81fc-44b1-8fbc-c4707b70a2f7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:11.850261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff40b60-81fc-44b1-8fbc-c4707b70a2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:11.850530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff40b60-81fc-44b1-8fbc-c4707b70a2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:11.917791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4ff40b60-81fc-44b1-8fbc-c4707b70a2f7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:11.918931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ff40b60-81fc-44b1-8fbc-c4707b70a2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:11.929114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ce523-dfbb-471b-baae-eff2d9615ee6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:11.929730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ce523-dfbb-471b-baae-eff2d9615ee6 service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:11.930005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ce523-dfbb-471b-baae-eff2d9615ee6 service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:11.930273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ce523-dfbb-471b-baae-eff2d9615ee6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:11.931938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-396ce523-dfbb-471b-baae-eff2d9615ee6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:11.939228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:11.994920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:12.017852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ce523-dfbb-471b-baae-eff2d9615ee6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:12.018404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 692/1401] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:12.019839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ce523-dfbb-471b-baae-eff2d9615ee6 service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:12.027949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 710/1402] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:28:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:28:12.307275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff40b60-81fc-44b1-8fbc-c4707b70a2f7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:12.308627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ff40b60-81fc-44b1-8fbc-c4707b70a2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:12.445250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff40b60-81fc-44b1-8fbc-c4707b70a2f7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:12.542828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:12.575208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd88064-6433-414f-88f5-d0df43c6f254 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:12.575833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd88064-6433-414f-88f5-d0df43c6f254 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:12.576108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd88064-6433-414f-88f5-d0df43c6f254 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:12.576377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd88064-6433-414f-88f5-d0df43c6f254 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:12.577988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2bd88064-6433-414f-88f5-d0df43c6f254 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:12.677072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd88064-6433-414f-88f5-d0df43c6f254 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:12.680900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd88064-6433-414f-88f5-d0df43c6f254 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:12.702206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 693/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:14.659406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a078-3b33-4153-9866-444f68049bac None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:14.660166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a078-3b33-4153-9866-444f68049bac None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:14.660409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a078-3b33-4153-9866-444f68049bac None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:14.661768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 711/1404] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:14.667092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aa2d1-4b72-4d08-bb05-719c4084b0fb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:14.667554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aa2d1-4b72-4d08-bb05-719c4084b0fb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:14.667646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aa2d1-4b72-4d08-bb05-719c4084b0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:14.739168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e88aa2d1-4b72-4d08-bb05-719c4084b0fb None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:14.740326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e88aa2d1-4b72-4d08-bb05-719c4084b0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:14.795819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:14.843847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 694/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:28:15.127769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a078-3b33-4153-9866-444f68049bac None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:15.129383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7c97a078-3b33-4153-9866-444f68049bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:15.294811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a078-3b33-4153-9866-444f68049bac None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:15.298108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:15.421653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a38a8c-4adc-4689-b7a4-ac46ea6b23b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:15.422179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a38a8c-4adc-4689-b7a4-ac46ea6b23b8 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:15.422459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a38a8c-4adc-4689-b7a4-ac46ea6b23b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:15.422750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a38a8c-4adc-4689-b7a4-ac46ea6b23b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:15.424379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-93a38a8c-4adc-4689-b7a4-ac46ea6b23b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:15.532595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a38a8c-4adc-4689-b7a4-ac46ea6b23b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:15.534250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a38a8c-4adc-4689-b7a4-ac46ea6b23b8 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:15.553599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 712/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:17.522837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1bc16-5d25-45e4-814d-425d2b9953df None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:17.523381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1bc16-5d25-45e4-814d-425d2b9953df None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:17.523381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1bc16-5d25-45e4-814d-425d2b9953df None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:17.524954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 695/1407] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:17.530156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a351c-680c-4613-af18-a61e153669c6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:17.530633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a351c-680c-4613-af18-a61e153669c6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:17.530781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a351c-680c-4613-af18-a61e153669c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:17.540932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:17.606908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-187a351c-680c-4613-af18-a61e153669c6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:17.608340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-187a351c-680c-4613-af18-a61e153669c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:17.715517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 713/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:18.011628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1bc16-5d25-45e4-814d-425d2b9953df None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:18.013381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-acf1bc16-5d25-45e4-814d-425d2b9953df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:18.187082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1bc16-5d25-45e4-814d-425d2b9953df None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:18.243416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:18.359571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647f7cb-a3e7-43e0-ab87-b26caaf5a6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:18.360224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647f7cb-a3e7-43e0-ab87-b26caaf5a6e4 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:18.360639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647f7cb-a3e7-43e0-ab87-b26caaf5a6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:18.360914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647f7cb-a3e7-43e0-ab87-b26caaf5a6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:18.362550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b647f7cb-a3e7-43e0-ab87-b26caaf5a6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:18.457619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647f7cb-a3e7-43e0-ab87-b26caaf5a6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:18.459215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647f7cb-a3e7-43e0-ab87-b26caaf5a6e4 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:18.480457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 696/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:27.713804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=62 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:28.480649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=82 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:34.204361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae5da-34b7-49a4-b971-7a31aaa9406b None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:34.204973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae5da-34b7-49a4-b971-7a31aaa9406b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:34.204973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae5da-34b7-49a4-b971-7a31aaa9406b None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:34.206622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 714/1410] 10.222.0.42 () {54 vars in 918 bytes} [Wed Aug 31 04:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:34.211991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf6de4-0810-4314-82ea-0c1c76ceaf07 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:34.212409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf6de4-0810-4314-82ea-0c1c76ceaf07 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:34.212502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf6de4-0810-4314-82ea-0c1c76ceaf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:34.295067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c6cf6de4-0810-4314-82ea-0c1c76ceaf07 None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:34.296289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c6cf6de4-0810-4314-82ea-0c1c76ceaf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:34.418786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 697/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Aug 31 04:28:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:34.433658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607563d-5f39-4538-b4c2-8a90d9a07c91 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:34.433943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607563d-5f39-4538-b4c2-8a90d9a07c91 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:34.434210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607563d-5f39-4538-b4c2-8a90d9a07c91 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:34.435590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 715/1412] 10.222.0.42 () {54 vars in 918 bytes} [Wed Aug 31 04:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:34.440444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e05a3-49e3-41b9-ac99-d8eadcb71b8a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:34.440731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e05a3-49e3-41b9-ac99-d8eadcb71b8a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:34.441002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e05a3-49e3-41b9-ac99-d8eadcb71b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:34.518428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:34.521821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-de5e05a3-49e3-41b9-ac99-d8eadcb71b8a None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:34.523535 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de5e05a3-49e3-41b9-ac99-d8eadcb71b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:34.658776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 698/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Aug 31 04:28:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:37.036984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0c6d6-947e-4217-9ce1-0c035ff323a5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:37.037838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0c6d6-947e-4217-9ce1-0c035ff323a5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:37.037838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0c6d6-947e-4217-9ce1-0c035ff323a5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:37.039988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:28:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:37.048824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c31f-1ffa-4f93-80fd-2f70d3b7189b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:37.049251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c31f-1ffa-4f93-80fd-2f70d3b7189b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:37.049388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c31f-1ffa-4f93-80fd-2f70d3b7189b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:37.119330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1876c31f-1ffa-4f93-80fd-2f70d3b7189b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:37.120538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1876c31f-1ffa-4f93-80fd-2f70d3b7189b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:37.180699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:37.226199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:37.515429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0c6d6-947e-4217-9ce1-0c035ff323a5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:37.518097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-67e0c6d6-947e-4217-9ce1-0c035ff323a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:37.679416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0c6d6-947e-4217-9ce1-0c035ff323a5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:37.756536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:37.800246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33552c-ef4b-413a-81ad-462794ab7db8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:37.800856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33552c-ef4b-413a-81ad-462794ab7db8 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:37.801130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33552c-ef4b-413a-81ad-462794ab7db8 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:37.801792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33552c-ef4b-413a-81ad-462794ab7db8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:37.803215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3b33552c-ef4b-413a-81ad-462794ab7db8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:37.907102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33552c-ef4b-413a-81ad-462794ab7db8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:37.908728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33552c-ef4b-413a-81ad-462794ab7db8 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:37.932820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 717/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:38.836895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32da38-6711-42c5-9969-67882d44f9e2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:38.837472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32da38-6711-42c5-9969-67882d44f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:38.837472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32da38-6711-42c5-9969-67882d44f9e2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:38.838852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:38.844924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12410eec-208d-421e-885f-9dfffa9dbe30 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:38.845673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12410eec-208d-421e-885f-9dfffa9dbe30 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:38.845838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12410eec-208d-421e-885f-9dfffa9dbe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:38.925022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-12410eec-208d-421e-885f-9dfffa9dbe30 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:38.926235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-12410eec-208d-421e-885f-9dfffa9dbe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:38.954173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:39.027288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:39.176986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32da38-6711-42c5-9969-67882d44f9e2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:39.178401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8b32da38-6711-42c5-9969-67882d44f9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:39.350758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32da38-6711-42c5-9969-67882d44f9e2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:39.405057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:39.495715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e2c2a-de3f-4de8-9efe-61fd2f7f20e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:39.496463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2c2a-de3f-4de8-9efe-61fd2f7f20e9 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:39.496843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2c2a-de3f-4de8-9efe-61fd2f7f20e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:39.497234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2c2a-de3f-4de8-9efe-61fd2f7f20e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:39.499776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4b0e2c2a-de3f-4de8-9efe-61fd2f7f20e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:39.605740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e2c2a-de3f-4de8-9efe-61fd2f7f20e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:39.607374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e2c2a-de3f-4de8-9efe-61fd2f7f20e9 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:39.627689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 701/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:40.492603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f3cc-7f11-4c30-a732-ddfe11e39923 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:40.493104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f3cc-7f11-4c30-a732-ddfe11e39923 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:40.493199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f3cc-7f11-4c30-a732-ddfe11e39923 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:40.495315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:28:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:40.501297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8b38-e756-41ce-8699-71e4e3f64733 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:40.501754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8b38-e756-41ce-8699-71e4e3f64733 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:40.501846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8b38-e756-41ce-8699-71e4e3f64733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:40.559418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:40.582472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-83fe8b38-e756-41ce-8699-71e4e3f64733 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:40.583757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83fe8b38-e756-41ce-8699-71e4e3f64733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:40.711257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:40.863014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f3cc-7f11-4c30-a732-ddfe11e39923 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:40.864917 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b5e0f3cc-7f11-4c30-a732-ddfe11e39923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:41.023768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f3cc-7f11-4c30-a732-ddfe11e39923 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:41.036282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:41.148313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a2db-7a57-42d1-b7d2-82b6cee1562c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:41.148929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a2db-7a57-42d1-b7d2-82b6cee1562c service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:41.149205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a2db-7a57-42d1-b7d2-82b6cee1562c service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:41.149474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a2db-7a57-42d1-b7d2-82b6cee1562c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:41.151136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3912a2db-7a57-42d1-b7d2-82b6cee1562c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:41.248510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912a2db-7a57-42d1-b7d2-82b6cee1562c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:41.250128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912a2db-7a57-42d1-b7d2-82b6cee1562c service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:41.269484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 720/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:42.134011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eea736-d7cb-41c7-bda7-2017525089c6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:42.134573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eea736-d7cb-41c7-bda7-2017525089c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:42.134573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eea736-d7cb-41c7-bda7-2017525089c6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:42.135949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:42.142118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42169-0001-4cef-96f6-bdeaed0c7491 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:42.142856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42169-0001-4cef-96f6-bdeaed0c7491 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:42.143032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42169-0001-4cef-96f6-bdeaed0c7491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:42.169414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:42.243255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cdb42169-0001-4cef-96f6-bdeaed0c7491 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:42.244472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cdb42169-0001-4cef-96f6-bdeaed0c7491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:42.357202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:42.510122 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eea736-d7cb-41c7-bda7-2017525089c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:42.511678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-45eea736-d7cb-41c7-bda7-2017525089c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:42.647281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:42.689158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eea736-d7cb-41c7-bda7-2017525089c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:42.811206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f5ccc-2862-47c0-bf80-14fb63695c32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:42.811822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f5ccc-2862-47c0-bf80-14fb63695c32 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:42.812098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f5ccc-2862-47c0-bf80-14fb63695c32 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:42.812367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f5ccc-2862-47c0-bf80-14fb63695c32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:42.813986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c39f5ccc-2862-47c0-bf80-14fb63695c32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:42.898076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:42.919331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f5ccc-2862-47c0-bf80-14fb63695c32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:42.920968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f5ccc-2862-47c0-bf80-14fb63695c32 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:42.941488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 704/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:43.798539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69a87a-cf28-4300-a70a-100a1af83269 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:43.799143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69a87a-cf28-4300-a70a-100a1af83269 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:43.799143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69a87a-cf28-4300-a70a-100a1af83269 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:43.800650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:28:43.806768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b24b-e120-4bd8-b03e-db9573360620 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:43.807535 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b24b-e120-4bd8-b03e-db9573360620 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:43.807663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b24b-e120-4bd8-b03e-db9573360620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:43.903117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6d57b24b-e120-4bd8-b03e-db9573360620 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:43.904363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d57b24b-e120-4bd8-b03e-db9573360620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:44.008458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:44.158954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69a87a-cf28-4300-a70a-100a1af83269 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:44.161453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7c69a87a-cf28-4300-a70a-100a1af83269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:44.256861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:44.326387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69a87a-cf28-4300-a70a-100a1af83269 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:44.453389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a251da-21ff-4f4e-a91c-42eaae43db53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:44.454006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a251da-21ff-4f4e-a91c-42eaae43db53 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:44.454281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a251da-21ff-4f4e-a91c-42eaae43db53 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:44.454555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a251da-21ff-4f4e-a91c-42eaae43db53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:44.456583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-95a251da-21ff-4f4e-a91c-42eaae43db53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:44.505297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:44.557124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a251da-21ff-4f4e-a91c-42eaae43db53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:44.558771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a251da-21ff-4f4e-a91c-42eaae43db53 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:44.578170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 723/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:45.382640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d603a8-d58a-4303-b443-9909676d0e06 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:45.383633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d603a8-d58a-4303-b443-9909676d0e06 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:45.383633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d603a8-d58a-4303-b443-9909676d0e06 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:45.385556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:28:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:45.391956 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13a4e8-c92c-4dee-b5d5-3c28b22c5da5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:45.392677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13a4e8-c92c-4dee-b5d5-3c28b22c5da5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:45.392859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13a4e8-c92c-4dee-b5d5-3c28b22c5da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:45.480900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fc13a4e8-c92c-4dee-b5d5-3c28b22c5da5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:45.482270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fc13a4e8-c92c-4dee-b5d5-3c28b22c5da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:45.582550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:45.740459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d603a8-d58a-4303-b443-9909676d0e06 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:45.742581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f0d603a8-d58a-4303-b443-9909676d0e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:45.796985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:45.909408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d603a8-d58a-4303-b443-9909676d0e06 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:46.035266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c553c3-3007-4951-b316-59c7890acc65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:46.035791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c553c3-3007-4951-b316-59c7890acc65 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:46.036065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c553c3-3007-4951-b316-59c7890acc65 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:46.036332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c553c3-3007-4951-b316-59c7890acc65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:46.037955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-14c553c3-3007-4951-b316-59c7890acc65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:46.045028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:46.134789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c553c3-3007-4951-b316-59c7890acc65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:46.136432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c553c3-3007-4951-b316-59c7890acc65 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:46.155282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 707/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:28:46.982245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c426f8-043e-4724-af2d-9dc1dd8af807 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:46.982900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c426f8-043e-4724-af2d-9dc1dd8af807 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:46.982900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c426f8-043e-4724-af2d-9dc1dd8af807 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:46.984899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:28:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:46.989882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151b521-b907-440e-9669-fd4bdd8b0e3c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:46.990317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151b521-b907-440e-9669-fd4bdd8b0e3c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:46.990430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151b521-b907-440e-9669-fd4bdd8b0e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:47.069915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3151b521-b907-440e-9669-fd4bdd8b0e3c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:47.071151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3151b521-b907-440e-9669-fd4bdd8b0e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:47.141284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:47.179239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:47.335334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c426f8-043e-4724-af2d-9dc1dd8af807 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:47.337828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f0c426f8-043e-4724-af2d-9dc1dd8af807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:47.365471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:47.514868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c426f8-043e-4724-af2d-9dc1dd8af807 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:47.623698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:47.651517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2ec1f-caf6-4142-be4b-1cd66054a7be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:47.652125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2ec1f-caf6-4142-be4b-1cd66054a7be service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:47.652396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2ec1f-caf6-4142-be4b-1cd66054a7be service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:47.652662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2ec1f-caf6-4142-be4b-1cd66054a7be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:47.654264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-94f2ec1f-caf6-4142-be4b-1cd66054a7be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:47.750546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2ec1f-caf6-4142-be4b-1cd66054a7be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:47.752227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2ec1f-caf6-4142-be4b-1cd66054a7be service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:47.771458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 726/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:48.589531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3c3a-7003-4b23-9e9b-23698670afbc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:48.590157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3c3a-7003-4b23-9e9b-23698670afbc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:48.590157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3c3a-7003-4b23-9e9b-23698670afbc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:48.591465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:48.597633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab333c-01c9-4df4-aaaa-f12603aa594c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:48.598315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab333c-01c9-4df4-aaaa-f12603aa594c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:48.598670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab333c-01c9-4df4-aaaa-f12603aa594c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:48.683495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7aab333c-01c9-4df4-aaaa-f12603aa594c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:48.684723 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7aab333c-01c9-4df4-aaaa-f12603aa594c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:48.728170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:48.789885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:48.947662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3c3a-7003-4b23-9e9b-23698670afbc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:48.949224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d6c3c3a-7003-4b23-9e9b-23698670afbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:49.104688 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3c3a-7003-4b23-9e9b-23698670afbc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:49.162666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:49.227534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc323e2-bbd7-4ede-9c1a-138971302484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:49.228144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc323e2-bbd7-4ede-9c1a-138971302484 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:49.228417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc323e2-bbd7-4ede-9c1a-138971302484 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:49.228650 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc323e2-bbd7-4ede-9c1a-138971302484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:49.230326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bfc323e2-bbd7-4ede-9c1a-138971302484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:49.325507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc323e2-bbd7-4ede-9c1a-138971302484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:49.327052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc323e2-bbd7-4ede-9c1a-138971302484 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:49.345895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 710/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:50.153721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5bd4b-7574-4f2f-9bb9-5d06a68ebb85 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:50.154292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5bd4b-7574-4f2f-9bb9-5d06a68ebb85 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:50.154292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5bd4b-7574-4f2f-9bb9-5d06a68ebb85 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:50.156395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:28:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:50.161460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95441142-3dcb-4865-9817-39a2abde828a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:50.161990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95441142-3dcb-4865-9817-39a2abde828a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:50.162569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95441142-3dcb-4865-9817-39a2abde828a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:50.246571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-95441142-3dcb-4865-9817-39a2abde828a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:50.247817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-95441142-3dcb-4865-9817-39a2abde828a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:50.249856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:50.347755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:50.506196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5bd4b-7574-4f2f-9bb9-5d06a68ebb85 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:50.508713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0fc5bd4b-7574-4f2f-9bb9-5d06a68ebb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:50.669499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5bd4b-7574-4f2f-9bb9-5d06a68ebb85 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:50.697565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:50.807435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18561e66-5a38-4813-872a-d346785c0f3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:50.808074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18561e66-5a38-4813-872a-d346785c0f3b service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:50.808300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18561e66-5a38-4813-872a-d346785c0f3b service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:50.808594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18561e66-5a38-4813-872a-d346785c0f3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:50.810195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-18561e66-5a38-4813-872a-d346785c0f3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:50.906452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18561e66-5a38-4813-872a-d346785c0f3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:50.908031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18561e66-5a38-4813-872a-d346785c0f3b service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:50.926365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 729/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:52.736786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e88a7-46fc-4c26-b377-a3563844d0f0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:52.737320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e88a7-46fc-4c26-b377-a3563844d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:52.737320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e88a7-46fc-4c26-b377-a3563844d0f0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:52.739401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:28:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:52.745339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed298-d168-4570-900d-ac4cb2dde808 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:52.745746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed298-d168-4570-900d-ac4cb2dde808 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:52.745885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed298-d168-4570-900d-ac4cb2dde808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:52.795915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:52.824314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-746ed298-d168-4570-900d-ac4cb2dde808 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:52.825540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-746ed298-d168-4570-900d-ac4cb2dde808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:52.933490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:53.227584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e88a7-46fc-4c26-b377-a3563844d0f0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:53.229127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8e6e88a7-46fc-4c26-b377-a3563844d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:53.341973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e88a7-46fc-4c26-b377-a3563844d0f0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:53.383422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:53.427591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b9855-69d4-4141-8842-446c7c639c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:53.428310 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b9855-69d4-4141-8842-446c7c639c57 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:53.428584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b9855-69d4-4141-8842-446c7c639c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:53.428847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b9855-69d4-4141-8842-446c7c639c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:53.430479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5f8b9855-69d4-4141-8842-446c7c639c57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:53.549741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b9855-69d4-4141-8842-446c7c639c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:53.551219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b9855-69d4-4141-8842-446c7c639c57 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:53.571524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 713/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:55.628342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecfe74-18a4-4580-9328-c0f737f54401 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:55.628942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecfe74-18a4-4580-9328-c0f737f54401 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:55.628942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecfe74-18a4-4580-9328-c0f737f54401 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:55.630347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:55.636681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e405-468f-45c7-ad76-2f78a480728e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:55.637142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e405-468f-45c7-ad76-2f78a480728e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:55.637248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e405-468f-45c7-ad76-2f78a480728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:55.710171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-83d8e405-468f-45c7-ad76-2f78a480728e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:55.711404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83d8e405-468f-45c7-ad76-2f78a480728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:55.724470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:55.812546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:56.105570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecfe74-18a4-4580-9328-c0f737f54401 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:56.108111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-46ecfe74-18a4-4580-9328-c0f737f54401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:56.228724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecfe74-18a4-4580-9328-c0f737f54401 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:56.241546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:56.308062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785b056-b416-4076-82fc-5b11205792ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:56.308661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785b056-b416-4076-82fc-5b11205792ed service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:56.308936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785b056-b416-4076-82fc-5b11205792ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:56.309201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785b056-b416-4076-82fc-5b11205792ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:56.310856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2785b056-b416-4076-82fc-5b11205792ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:56.436618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785b056-b416-4076-82fc-5b11205792ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:56.437981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785b056-b416-4076-82fc-5b11205792ed service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:56.457565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 732/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:28:58.766611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bd48-f886-4218-8f03-fa9151ccba75 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:58.767479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bd48-f886-4218-8f03-fa9151ccba75 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:58.767479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bd48-f886-4218-8f03-fa9151ccba75 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:58.769664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:28:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:28:58.776152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59c5d-af6c-46ae-a4e6-22f4b8aa57a0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:58.776872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59c5d-af6c-46ae-a4e6-22f4b8aa57a0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:58.777033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59c5d-af6c-46ae-a4e6-22f4b8aa57a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:58.837348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:58.856335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ded59c5d-af6c-46ae-a4e6-22f4b8aa57a0 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:28:58.857550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ded59c5d-af6c-46ae-a4e6-22f4b8aa57a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:58.960098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:28:59.243058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6bd48-f886-4218-8f03-fa9151ccba75 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:28:59.245515 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b6d6bd48-f886-4218-8f03-fa9151ccba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:59.362103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6bd48-f886-4218-8f03-fa9151ccba75 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:28:59.402629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:28:59.436162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a90907-b78a-4162-92c2-7703631eec2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:28:59.436765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a90907-b78a-4162-92c2-7703631eec2c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:28:59.437035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a90907-b78a-4162-92c2-7703631eec2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:28:59.437305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a90907-b78a-4162-92c2-7703631eec2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:28:59.438916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-53a90907-b78a-4162-92c2-7703631eec2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:28:59.548266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a90907-b78a-4162-92c2-7703631eec2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:28:59.549684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a90907-b78a-4162-92c2-7703631eec2c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:28:59.571400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 716/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:01.574110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf4cd-870d-42e3-b9e3-a072db5c25e3 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:01.574678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf4cd-870d-42e3-b9e3-a072db5c25e3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:01.574678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf4cd-870d-42e3-b9e3-a072db5c25e3 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:01.576042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:01.581862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e5e19-eb95-48d2-8094-e9b0bf13298b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:01.582284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e5e19-eb95-48d2-8094-e9b0bf13298b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:01.582395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e5e19-eb95-48d2-8094-e9b0bf13298b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:01.653103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-530e5e19-eb95-48d2-8094-e9b0bf13298b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:01.654372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-530e5e19-eb95-48d2-8094-e9b0bf13298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:01.679461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:01.772934 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:02.056886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbf4cd-870d-42e3-b9e3-a072db5c25e3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:02.058481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f4fbf4cd-870d-42e3-b9e3-a072db5c25e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:02.186523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbf4cd-870d-42e3-b9e3-a072db5c25e3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:02.209050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:02.265177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275a651-ff9b-45e7-b3ce-5be14aecef25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:02.265791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a651-ff9b-45e7-b3ce-5be14aecef25 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:02.266069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a651-ff9b-45e7-b3ce-5be14aecef25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:02.266334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a651-ff9b-45e7-b3ce-5be14aecef25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:02.267972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6275a651-ff9b-45e7-b3ce-5be14aecef25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:02.380472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275a651-ff9b-45e7-b3ce-5be14aecef25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:02.381946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275a651-ff9b-45e7-b3ce-5be14aecef25 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:02.401158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 735/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:04.385823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae472-50f4-40a1-b88a-d09013b63720 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:04.386389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae472-50f4-40a1-b88a-d09013b63720 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:04.386389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae472-50f4-40a1-b88a-d09013b63720 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:04.387700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:04.393448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef0b1b-da36-4c13-a18e-c6969f4ce5ed None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:04.393907 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef0b1b-da36-4c13-a18e-c6969f4ce5ed None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:04.394004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef0b1b-da36-4c13-a18e-c6969f4ce5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:04.459648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:04.462940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d4ef0b1b-da36-4c13-a18e-c6969f4ce5ed None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:04.464638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d4ef0b1b-da36-4c13-a18e-c6969f4ce5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:04.572214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:04.854435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae472-50f4-40a1-b88a-d09013b63720 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:04.856896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dfcae472-50f4-40a1-b88a-d09013b63720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:04.973349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae472-50f4-40a1-b88a-d09013b63720 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:05.029096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:05.056869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18586c1-0344-41c3-9781-25e3ac41d1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:05.057517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18586c1-0344-41c3-9781-25e3ac41d1ba service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:05.057814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18586c1-0344-41c3-9781-25e3ac41d1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:05.058107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18586c1-0344-41c3-9781-25e3ac41d1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:05.059783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e18586c1-0344-41c3-9781-25e3ac41d1ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:05.173779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18586c1-0344-41c3-9781-25e3ac41d1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:05.175222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18586c1-0344-41c3-9781-25e3ac41d1ba service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:05.193311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 719/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:07.281155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4cb69-14ff-4f2e-8ae2-ded611a488e2 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:07.281782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4cb69-14ff-4f2e-8ae2-ded611a488e2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:07.281782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4cb69-14ff-4f2e-8ae2-ded611a488e2 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:07.283134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:07.291362 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77885fc-0597-4fea-bffb-21746aa7a082 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:07.291842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77885fc-0597-4fea-bffb-21746aa7a082 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:07.291951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77885fc-0597-4fea-bffb-21746aa7a082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:07.362929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d77885fc-0597-4fea-bffb-21746aa7a082 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:07.364052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d77885fc-0597-4fea-bffb-21746aa7a082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:07.393610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:07.473875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:07.757853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4cb69-14ff-4f2e-8ae2-ded611a488e2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:07.761061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0fa4cb69-14ff-4f2e-8ae2-ded611a488e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:07.890218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4cb69-14ff-4f2e-8ae2-ded611a488e2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:07.922554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:07.969226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1379a31-2d59-47ed-bb37-746b0646502b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:07.969823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1379a31-2d59-47ed-bb37-746b0646502b service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:07.970093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1379a31-2d59-47ed-bb37-746b0646502b service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:07.970365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1379a31-2d59-47ed-bb37-746b0646502b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:07.971996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f1379a31-2d59-47ed-bb37-746b0646502b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:08.082754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1379a31-2d59-47ed-bb37-746b0646502b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:08.084220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1379a31-2d59-47ed-bb37-746b0646502b service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:08.103951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 738/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:10.074462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286942-fdd8-4337-bc1c-d334ef7292c5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:10.075006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286942-fdd8-4337-bc1c-d334ef7292c5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:10.075006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286942-fdd8-4337-bc1c-d334ef7292c5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:10.076422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:10.085965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea53ee-c447-46df-89dd-2c0c607e1825 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:10.086694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea53ee-c447-46df-89dd-2c0c607e1825 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:10.086843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea53ee-c447-46df-89dd-2c0c607e1825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:10.165003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-19ea53ee-c447-46df-89dd-2c0c607e1825 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:10.166499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-19ea53ee-c447-46df-89dd-2c0c607e1825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:10.175701 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:10.271444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:10.551786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03286942-fdd8-4337-bc1c-d334ef7292c5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:10.554175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-03286942-fdd8-4337-bc1c-d334ef7292c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:10.679001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03286942-fdd8-4337-bc1c-d334ef7292c5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:10.742085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:10.762008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35911c3d-3467-4422-ae56-d4b715627bcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:10.762624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35911c3d-3467-4422-ae56-d4b715627bcb service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:10.762930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35911c3d-3467-4422-ae56-d4b715627bcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:10.763203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35911c3d-3467-4422-ae56-d4b715627bcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:10.764803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-35911c3d-3467-4422-ae56-d4b715627bcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:10.914696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35911c3d-3467-4422-ae56-d4b715627bcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:10.916780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35911c3d-3467-4422-ae56-d4b715627bcb service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:10.938937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 722/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:12.888827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec45f9-81f0-47e7-ae3a-6d8f9dc2fd75 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:12.889744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec45f9-81f0-47e7-ae3a-6d8f9dc2fd75 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:12.889744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec45f9-81f0-47e7-ae3a-6d8f9dc2fd75 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:12.892176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:29:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:12.901288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596a5b-fdc3-4c35-81bf-ab35206e55d6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:12.901714 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596a5b-fdc3-4c35-81bf-ab35206e55d6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:12.901805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54596a5b-fdc3-4c35-81bf-ab35206e55d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:12.980835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-54596a5b-fdc3-4c35-81bf-ab35206e55d6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:12.982625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-54596a5b-fdc3-4c35-81bf-ab35206e55d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:13.025765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:13.090946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:13.372729 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec45f9-81f0-47e7-ae3a-6d8f9dc2fd75 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:13.375206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3dec45f9-81f0-47e7-ae3a-6d8f9dc2fd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:13.492381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec45f9-81f0-47e7-ae3a-6d8f9dc2fd75 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:13.535266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:13.577236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b64522-ac66-498a-96e9-b9091ebe2368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:13.577850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b64522-ac66-498a-96e9-b9091ebe2368 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:13.578126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b64522-ac66-498a-96e9-b9091ebe2368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:13.578394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b64522-ac66-498a-96e9-b9091ebe2368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:13.580085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-65b64522-ac66-498a-96e9-b9091ebe2368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:13.704558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b64522-ac66-498a-96e9-b9091ebe2368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:13.705923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b64522-ac66-498a-96e9-b9091ebe2368 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:13.726091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 741/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:15.619299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5b5eb-569e-41ef-9484-f4716d65967c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:15.620587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5b5eb-569e-41ef-9484-f4716d65967c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:15.620587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5b5eb-569e-41ef-9484-f4716d65967c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:15.622463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:29:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:15.631498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cccbfa-df51-4db5-a93c-15e74977b564 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:15.632280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cccbfa-df51-4db5-a93c-15e74977b564 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:15.632484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cccbfa-df51-4db5-a93c-15e74977b564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:15.708212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a9cccbfa-df51-4db5-a93c-15e74977b564 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:15.709433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a9cccbfa-df51-4db5-a93c-15e74977b564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:15.728365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:15.812183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:16.104517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5b5eb-569e-41ef-9484-f4716d65967c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:16.106063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e1e5b5eb-569e-41ef-9484-f4716d65967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:16.340034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5b5eb-569e-41ef-9484-f4716d65967c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:16.409185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:16.417332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c214e6-aa50-4c46-bb67-7d3ecc44bcce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:16.417944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c214e6-aa50-4c46-bb67-7d3ecc44bcce service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:16.418215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c214e6-aa50-4c46-bb67-7d3ecc44bcce service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:16.418608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c214e6-aa50-4c46-bb67-7d3ecc44bcce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:16.420308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10c214e6-aa50-4c46-bb67-7d3ecc44bcce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:16.533401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c214e6-aa50-4c46-bb67-7d3ecc44bcce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:16.534864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c214e6-aa50-4c46-bb67-7d3ecc44bcce service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:16.559126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 725/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:18.471773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e5c8d-f421-442c-9b33-4b4d01c0a81d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:18.472259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e5c8d-f421-442c-9b33-4b4d01c0a81d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:18.472471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e5c8d-f421-442c-9b33-4b4d01c0a81d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:18.473685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:18.480397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d5789-8fa6-409c-a6a6-e6eec8183206 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:18.481141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d5789-8fa6-409c-a6a6-e6eec8183206 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:18.481334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d5789-8fa6-409c-a6a6-e6eec8183206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:18.553323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8f4d5789-8fa6-409c-a6a6-e6eec8183206 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:18.554572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f4d5789-8fa6-409c-a6a6-e6eec8183206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:18.618063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:18.678811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:18.959352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e5c8d-f421-442c-9b33-4b4d01c0a81d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:18.960905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-611e5c8d-f421-442c-9b33-4b4d01c0a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:19.075044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e5c8d-f421-442c-9b33-4b4d01c0a81d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:19.124434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:19.160088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed0b89-bcd8-4e05-92e3-19f2a31767d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:19.160699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed0b89-bcd8-4e05-92e3-19f2a31767d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:19.160970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed0b89-bcd8-4e05-92e3-19f2a31767d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:19.161234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed0b89-bcd8-4e05-92e3-19f2a31767d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:19.162855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b4ed0b89-bcd8-4e05-92e3-19f2a31767d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:19.278646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed0b89-bcd8-4e05-92e3-19f2a31767d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:19.280023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed0b89-bcd8-4e05-92e3-19f2a31767d2 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:19.298190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 744/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:28.676749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=22 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:29.296340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=66 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:38.801871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bab05-80a5-406a-bf5a-03a9ae1a409a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:38.802763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bab05-80a5-406a-bf5a-03a9ae1a409a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:38.802763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0bab05-80a5-406a-bf5a-03a9ae1a409a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:38.804825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 727/1471] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:29:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:38.810598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f336d6-9898-4b13-84de-560dcf383b77 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:38.811078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f336d6-9898-4b13-84de-560dcf383b77 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:38.811191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f336d6-9898-4b13-84de-560dcf383b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:38.878758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c5f336d6-9898-4b13-84de-560dcf383b77 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:38.879987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c5f336d6-9898-4b13-84de-560dcf383b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:38.981955 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 745/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:39.280736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f71c-7c78-4506-bb79-143664b82a18 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:39.281119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f71c-7c78-4506-bb79-143664b82a18 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:39.281393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f71c-7c78-4506-bb79-143664b82a18 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:39.283590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 728/1473] 10.222.0.42 () {54 vars in 967 bytes} [Wed Aug 31 04:29:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:39.288615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a407e9-b361-4842-9735-5949f3198c17 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:39.288895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a407e9-b361-4842-9735-5949f3198c17 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:39.289164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a407e9-b361-4842-9735-5949f3198c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:39.373503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a0a407e9-b361-4842-9735-5949f3198c17 None None] MFA Rules not processed for user `ca15f81281dc42f4befeb9335698d18f`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:39.374669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a0a407e9-b361-4842-9735-5949f3198c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:39.397297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:39.468508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 746/1474] 10.222.0.42 () {58 vars in 1064 bytes} [Wed Aug 31 04:29:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:29:39.476353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f71c-7c78-4506-bb79-143664b82a18 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:39.477900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aed7f71c-7c78-4506-bb79-143664b82a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:39.603925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f71c-7c78-4506-bb79-143664b82a18 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:39.689120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413ae56-449f-46e7-8128-08dac4d8be3a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:39.689651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413ae56-449f-46e7-8128-08dac4d8be3a service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:39.689922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413ae56-449f-46e7-8128-08dac4d8be3a service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:39.690212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413ae56-449f-46e7-8128-08dac4d8be3a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:39.691956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8413ae56-449f-46e7-8128-08dac4d8be3a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:39.749717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:39.806429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413ae56-449f-46e7-8128-08dac4d8be3a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:39.807889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413ae56-449f-46e7-8128-08dac4d8be3a service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:39.826825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 729/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:41.808046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ff7f4-215a-4007-aeda-7d488ec074cc None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:41.809245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ff7f4-215a-4007-aeda-7d488ec074cc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:41.809245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ff7f4-215a-4007-aeda-7d488ec074cc None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:41.811185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 747/1476] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:29:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:41.817461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa0b83-96aa-45d4-9d95-b5deb1d562c2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:41.817929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa0b83-96aa-45d4-9d95-b5deb1d562c2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:41.818026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa0b83-96aa-45d4-9d95-b5deb1d562c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:41.896292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8faa0b83-96aa-45d4-9d95-b5deb1d562c2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:41.897493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8faa0b83-96aa-45d4-9d95-b5deb1d562c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:42.000373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 730/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:42.272445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ff7f4-215a-4007-aeda-7d488ec074cc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:42.275021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-919ff7f4-215a-4007-aeda-7d488ec074cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:42.430748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ff7f4-215a-4007-aeda-7d488ec074cc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:42.485099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:42.534542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002aa6d-0cee-4fe5-80ee-721a109ea86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:42.535189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002aa6d-0cee-4fe5-80ee-721a109ea86f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:42.535458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002aa6d-0cee-4fe5-80ee-721a109ea86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:42.535724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002aa6d-0cee-4fe5-80ee-721a109ea86f admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:42.536368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002aa6d-0cee-4fe5-80ee-721a109ea86f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:42.538183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-0002aa6d-0cee-4fe5-80ee-721a109ea86f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:29:42.538183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Aug 31 04:29:42.538183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: Aug 31 04:29:42.539375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002aa6d-0cee-4fe5-80ee-721a109ea86f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:42.558205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 748/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Aug 31 04:29:42 2022] GET /identity/v3/projects => generated 2553 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:29:44.492729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916601b2-d51d-4a0a-a3bb-ecc5f43ec46a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:44.493192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916601b2-d51d-4a0a-a3bb-ecc5f43ec46a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:44.493299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916601b2-d51d-4a0a-a3bb-ecc5f43ec46a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:44.494593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 731/1479] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:44.501028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7fa2-fec3-4a66-9680-3246c25464ac None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:44.501719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7fa2-fec3-4a66-9680-3246c25464ac None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:44.501922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7fa2-fec3-4a66-9680-3246c25464ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:44.573454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-559b7fa2-fec3-4a66-9680-3246c25464ac None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:44.574751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-559b7fa2-fec3-4a66-9680-3246c25464ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:44.676425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 749/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:44.964162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916601b2-d51d-4a0a-a3bb-ecc5f43ec46a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:44.965869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-916601b2-d51d-4a0a-a3bb-ecc5f43ec46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:44.976301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:45.125298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916601b2-d51d-4a0a-a3bb-ecc5f43ec46a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:45.210317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:45.245319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15068704-725d-4e60-9b94-671308b8ebee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:45.246332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15068704-725d-4e60-9b94-671308b8ebee service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:45.246640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15068704-725d-4e60-9b94-671308b8ebee service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:45.246938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15068704-725d-4e60-9b94-671308b8ebee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:45.248540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-15068704-725d-4e60-9b94-671308b8ebee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:45.345216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15068704-725d-4e60-9b94-671308b8ebee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:45.346794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15068704-725d-4e60-9b94-671308b8ebee service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:45.365375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 732/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:47.459197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1e991-caa8-47aa-8487-67c373cc4725 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:47.460107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1e991-caa8-47aa-8487-67c373cc4725 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:47.460107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1e991-caa8-47aa-8487-67c373cc4725 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:47.462185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 750/1482] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:29:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:47.468556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64321f-3918-4811-a748-e90a49fd4397 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:47.469284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64321f-3918-4811-a748-e90a49fd4397 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:47.469482 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64321f-3918-4811-a748-e90a49fd4397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:47.542245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cf64321f-3918-4811-a748-e90a49fd4397 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:47.543617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cf64321f-3918-4811-a748-e90a49fd4397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:47.591724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:47.647267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 733/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:47.932411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1e991-caa8-47aa-8487-67c373cc4725 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:47.934728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d9f1e991-caa8-47aa-8487-67c373cc4725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:47.961755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:48.044552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1e991-caa8-47aa-8487-67c373cc4725 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:48.119756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23239cb1-206b-4c14-91b6-9aa315210ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:48.120353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23239cb1-206b-4c14-91b6-9aa315210ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:48.120645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23239cb1-206b-4c14-91b6-9aa315210ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:48.120933 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23239cb1-206b-4c14-91b6-9aa315210ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:48.122506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-23239cb1-206b-4c14-91b6-9aa315210ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:48.198918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:48.240249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23239cb1-206b-4c14-91b6-9aa315210ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:48.241706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23239cb1-206b-4c14-91b6-9aa315210ee7 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:48.260478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 751/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:50.225095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f241d-f2ec-412f-9ea8-071d3934d8e4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:50.225866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f241d-f2ec-412f-9ea8-071d3934d8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:50.226025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f241d-f2ec-412f-9ea8-071d3934d8e4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:50.228380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 734/1485] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:29:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:50.235020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1952cf-ba55-4c72-ae73-b37d421d1026 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:50.235500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1952cf-ba55-4c72-ae73-b37d421d1026 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:50.235605 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1952cf-ba55-4c72-ae73-b37d421d1026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:50.308752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7e1952cf-ba55-4c72-ae73-b37d421d1026 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:50.309947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7e1952cf-ba55-4c72-ae73-b37d421d1026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:50.411203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 752/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:50.699071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f241d-f2ec-412f-9ea8-071d3934d8e4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:50.702101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-151f241d-f2ec-412f-9ea8-071d3934d8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:50.828350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f241d-f2ec-412f-9ea8-071d3934d8e4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:50.911511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:50.913259 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3424eb7-4bdd-4528-b757-b03688d6b8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:50.913827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3424eb7-4bdd-4528-b757-b03688d6b8ce service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:50.914211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3424eb7-4bdd-4528-b757-b03688d6b8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:50.914375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3424eb7-4bdd-4528-b757-b03688d6b8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:50.916632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e3424eb7-4bdd-4528-b757-b03688d6b8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:51.028553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3424eb7-4bdd-4528-b757-b03688d6b8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:51.030021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3424eb7-4bdd-4528-b757-b03688d6b8ce service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:51.048332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 735/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:29:53.101731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2850eeb-6ecb-45fc-a3e8-a646772aa5a2 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:53.102277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2850eeb-6ecb-45fc-a3e8-a646772aa5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:53.102277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2850eeb-6ecb-45fc-a3e8-a646772aa5a2 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:53.103673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 753/1488] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:53.109530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0193cb-09c5-49b1-be3d-d0a699bcd040 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:53.109992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0193cb-09c5-49b1-be3d-d0a699bcd040 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:53.110118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0193cb-09c5-49b1-be3d-d0a699bcd040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:53.180633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4c0193cb-09c5-49b1-be3d-d0a699bcd040 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:53.181849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c0193cb-09c5-49b1-be3d-d0a699bcd040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:53.241313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:53.288070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 736/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:53.573935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2850eeb-6ecb-45fc-a3e8-a646772aa5a2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:53.575839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b2850eeb-6ecb-45fc-a3e8-a646772aa5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:53.613477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:53.686157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2850eeb-6ecb-45fc-a3e8-a646772aa5a2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:53.776118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195871-1854-4785-b611-ba3478a85aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:53.776763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195871-1854-4785-b611-ba3478a85aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:53.777048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195871-1854-4785-b611-ba3478a85aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:53.777517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195871-1854-4785-b611-ba3478a85aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:53.779184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-33195871-1854-4785-b611-ba3478a85aa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:53.878825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:53.909382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33195871-1854-4785-b611-ba3478a85aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:53.910874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33195871-1854-4785-b611-ba3478a85aa9 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:53.934766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 754/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:29:56.083809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf82a2-c86c-45bb-ab42-064b72cf5bc3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:56.084297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf82a2-c86c-45bb-ab42-064b72cf5bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:56.084297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf82a2-c86c-45bb-ab42-064b72cf5bc3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:56.085699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 737/1491] 10.222.0.42 () {54 vars in 927 bytes} [Wed Aug 31 04:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:29:56.092447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f29cd8-21b5-4a6e-b837-8e6e7cd86520 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:56.092908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f29cd8-21b5-4a6e-b837-8e6e7cd86520 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:56.093016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f29cd8-21b5-4a6e-b837-8e6e7cd86520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:56.162966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-32f29cd8-21b5-4a6e-b837-8e6e7cd86520 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:29:56.164204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-32f29cd8-21b5-4a6e-b837-8e6e7cd86520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:56.268219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 755/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Aug 31 04:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:29:56.566074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf82a2-c86c-45bb-ab42-064b72cf5bc3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:29:56.568469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-50cf82a2-c86c-45bb-ab42-064b72cf5bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:56.698264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf82a2-c86c-45bb-ab42-064b72cf5bc3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:29:56.777191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373252e6-5cd4-45b7-9993-2df85d1d999a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:29:56.777720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373252e6-5cd4-45b7-9993-2df85d1d999a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:29:56.778057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373252e6-5cd4-45b7-9993-2df85d1d999a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:29:56.778326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373252e6-5cd4-45b7-9993-2df85d1d999a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:29:56.779953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-373252e6-5cd4-45b7-9993-2df85d1d999a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:29:56.789788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:29:56.907931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373252e6-5cd4-45b7-9993-2df85d1d999a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:29:56.909363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373252e6-5cd4-45b7-9993-2df85d1d999a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:29:56.928208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 738/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:29:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:30:06.266368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=82 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:30:06.925759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=50 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:09.278613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf868035-5cc7-4d90-92b2-cf193a3984db None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:09.279547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf868035-5cc7-4d90-92b2-cf193a3984db None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:09.279547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf868035-5cc7-4d90-92b2-cf193a3984db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:09.375091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cf868035-5cc7-4d90-92b2-cf193a3984db None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:31:09.376281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cf868035-5cc7-4d90-92b2-cf193a3984db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:09.479930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 756/1494] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:31:09.488363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95048c-9bb0-42ca-9d57-149fdb2632a5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:09.488878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95048c-9bb0-42ca-9d57-149fdb2632a5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:09.488878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95048c-9bb0-42ca-9d57-149fdb2632a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:09.568029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6c95048c-9bb0-42ca-9d57-149fdb2632a5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:31:09.569224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c95048c-9bb0-42ca-9d57-149fdb2632a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:09.673628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 739/1495] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:31:09.678602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf868035-5cc7-4d90-92b2-cf193a3984db None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:09.679915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cf868035-5cc7-4d90-92b2-cf193a3984db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:09.693120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf868035-5cc7-4d90-92b2-cf193a3984db None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:09.886073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:09.923611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:09.924277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:09.924527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:09.924792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:09.925476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:31:09.925775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:09.926917 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:31:09.926917 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Aug 31 04:31:09.926917 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: Aug 31 04:31:09.927815 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:09.940082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 757/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:31:09.949105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95048c-9bb0-42ca-9d57-149fdb2632a5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:09.959107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95048c-9bb0-42ca-9d57-149fdb2632a5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:10.024417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:10.057776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:10.058382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:10.058654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:10.058945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:10.059570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:10.061297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:31:10.061297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 31 04:31:10.061297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: Aug 31 04:31:10.062484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:10.121830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 740/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:31:09 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:31:10.126904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:10.128630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:10.164335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de3c02-e142-4d2f-bb30-4620f05e90cf admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:10.276544 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:10.277182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:10.277458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:10.277722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:10.278362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:10.279437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:31:10.279437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Aug 31 04:31:10.279437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: Aug 31 04:31:10.280665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:10.327689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 758/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:31:10 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:31:10.332572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:10.334076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:10.403033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f82fa-76c8-44dd-9d60-163ba45c3334 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:10.529713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:10.548134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:10.549031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:10.549395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:10.549697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:10.550358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:10.551450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:31:10.551450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 31 04:31:10.551450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: Aug 31 04:31:10.552399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:10.567890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 741/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:31:10.572788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:10.574336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:10.587519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354d883-e348-460b-8957-ee5054d6b6c6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:10.624249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:10.726001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830eda6-e7d1-4fd6-9c5d-8d6598745675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:10.726581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830eda6-e7d1-4fd6-9c5d-8d6598745675 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:10.727003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830eda6-e7d1-4fd6-9c5d-8d6598745675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:10.727268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830eda6-e7d1-4fd6-9c5d-8d6598745675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:10.727966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830eda6-e7d1-4fd6-9c5d-8d6598745675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:10.728983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-3830eda6-e7d1-4fd6-9c5d-8d6598745675 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:31:10.728983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Aug 31 04:31:10.728983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: Aug 31 04:31:10.729995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830eda6-e7d1-4fd6-9c5d-8d6598745675 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:10.743247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 759/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:31:10.747745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:10.749283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:10.784204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375713af-d711-4300-ae80-aebc1827b41f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:10.894667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211b9ca-63e3-4443-b4a6-70e3036e80de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:10.895539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b9ca-63e3-4443-b4a6-70e3036e80de admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:10.895886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b9ca-63e3-4443-b4a6-70e3036e80de admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:10.896306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211b9ca-63e3-4443-b4a6-70e3036e80de admin admin] PATH_INFO: `/v3/projects/09e6a07a2b2f47db9b0f433968fbd1c1/users/3626c14d9c364ead932bd47634df13b7/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:10.920862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211b9ca-63e3-4443-b4a6-70e3036e80de admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e6a07a2b2f47db9b0f433968fbd1c1, user_id=3626c14d9c364ead932bd47634df13b7, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:10.921910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-6211b9ca-63e3-4443-b4a6-70e3036e80de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:31:10.921910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 31 04:31:10.921910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: Aug 31 04:31:10.923912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211b9ca-63e3-4443-b4a6-70e3036e80de admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:10.953724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 742/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:31:10 2022] PUT /identity/v3/projects/09e6a07a2b2f47db9b0f433968fbd1c1/users/3626c14d9c364ead932bd47634df13b7/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:31:10.966536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c06a-9882-46e2-a0e2-1ba64f3a27d0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:10.966858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c06a-9882-46e2-a0e2-1ba64f3a27d0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:10.967132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c06a-9882-46e2-a0e2-1ba64f3a27d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:11.028897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d3c9c06a-9882-46e2-a0e2-1ba64f3a27d0 None None] MFA Rules not processed for user `3626c14d9c364ead932bd47634df13b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:31:11.030105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d3c9c06a-9882-46e2-a0e2-1ba64f3a27d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:11.112865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 760/1502] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:31:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:31:11.127417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613477-2c5b-4f60-ab29-2199eda92e77 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:11.127704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613477-2c5b-4f60-ab29-2199eda92e77 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:11.127976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613477-2c5b-4f60-ab29-2199eda92e77 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:11.129400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 743/1503] 10.222.0.42 () {54 vars in 964 bytes} [Wed Aug 31 04:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:31:11.134346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef5a68-f737-4264-86b2-e54ffdb28634 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:11.134621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef5a68-f737-4264-86b2-e54ffdb28634 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:11.134947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef5a68-f737-4264-86b2-e54ffdb28634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:11.167525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:11.226757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5bef5a68-f737-4264-86b2-e54ffdb28634 None None] MFA Rules not processed for user `6f3a86b76cff4b48a382888cbac913f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:31:11.227988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5bef5a68-f737-4264-86b2-e54ffdb28634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:11.320692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 761/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Aug 31 04:31:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:31:11.333340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48613477-2c5b-4f60-ab29-2199eda92e77 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:11.336063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-48613477-2c5b-4f60-ab29-2199eda92e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:11.394997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:11.451813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48613477-2c5b-4f60-ab29-2199eda92e77 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:11.536003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:11.536718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:11.536840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:11.537111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:11.538702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:11.646988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:11.648936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:11.654092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:11.676094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 744/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:31:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:31:11.729811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef0b30-c2d1-41ec-a337-55d06f43f816 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:11.730088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef0b30-c2d1-41ec-a337-55d06f43f816 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:11.730358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef0b30-c2d1-41ec-a337-55d06f43f816 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:11.731658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 762/1506] 10.222.0.42 () {52 vars in 830 bytes} [Wed Aug 31 04:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:31:11.735731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:11.737211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:11.842065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72268025-f450-4599-ab9f-e7dfd5e21394 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:11.938641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:11.939240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:11.939516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:11.939783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] PATH_INFO: `/v3/users/3626c14d9c364ead932bd47634df13b7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:11.953719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3626c14d9c364ead932bd47634df13b7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:11.954864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:31:11.954864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 31 04:31:11.954864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: Aug 31 04:31:11.956076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:11.970419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:12.035217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3626c14d9c364ead932bd47634df13b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:12.047977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3626c14d9c364ead932bd47634df13b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:12.048615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3626c14d9c364ead932bd47634df13b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:12.073343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3626c14d9c364ead932bd47634df13b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:12.073824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3626c14d9c364ead932bd47634df13b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:12.094354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 745/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:31:11 2022] DELETE /identity/v3/users/3626c14d9c364ead932bd47634df13b7 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:31:12.099333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ace930-802e-4072-832d-642451e305c4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:12.099746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ace930-802e-4072-832d-642451e305c4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:12.099879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ace930-802e-4072-832d-642451e305c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:12.160206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-91ace930-802e-4072-832d-642451e305c4 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:31:12.161448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-91ace930-802e-4072-832d-642451e305c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:12.240263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:12.253867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 763/1508] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:31:12.264386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:12.265617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:12.368809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c925a53-6043-4d56-91df-645ef8e7e5dd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:12.427693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:12.452613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d0e09-438f-4d15-ae63-0868e1d15427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:12.453230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d0e09-438f-4d15-ae63-0868e1d15427 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:12.453511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d0e09-438f-4d15-ae63-0868e1d15427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:12.453777 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d0e09-438f-4d15-ae63-0868e1d15427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:12.455423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-890d0e09-438f-4d15-ae63-0868e1d15427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:12.570451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d0e09-438f-4d15-ae63-0868e1d15427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:12.571924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d0e09-438f-4d15-ae63-0868e1d15427 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:12.590661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 746/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:31:13.025048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ace930-802e-4072-832d-642451e305c4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:31:13.026307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-91ace930-802e-4072-832d-642451e305c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:31:13.039939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ace930-802e-4072-832d-642451e305c4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:31:13.144308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:31:13.144948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:31:13.145219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:31:13.145483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] PATH_INFO: `/v3/projects/09e6a07a2b2f47db9b0f433968fbd1c1` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:31:13.155431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e6a07a2b2f47db9b0f433968fbd1c1)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:31:13.156469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:31:13.156469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Aug 31 04:31:13.156469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: Aug 31 04:31:13.157659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:31:13.222251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e6a07a2b2f47db9b0f433968fbd1c1', 'request_id': 'req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:13.231928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e6a07a2b2f47db9b0f433968fbd1c1', 'request_id': 'req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:13.245757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e6a07a2b2f47db9b0f433968fbd1c1', 'request_id': 'req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:13.270371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] The token cache is being invalidate because project 09e6a07a2b2f47db9b0f433968fbd1c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:31:13.270654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:13.271179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-50fbf8a8-878a-48ba-ae3f-d7f642bbab9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:31:13.272597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 764/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:31:13 2022] DELETE /identity/v3/projects/09e6a07a2b2f47db9b0f433968fbd1c1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:31:22.588363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=62 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:31:23.268784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:32:42.289878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b24f8-4db5-4db7-af30-056e1cd9181d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:42.290507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b24f8-4db5-4db7-af30-056e1cd9181d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:42.290507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b24f8-4db5-4db7-af30-056e1cd9181d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:42.291831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:32:42.297891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3af88-4a00-4556-af54-480328ae5e6c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:42.298367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3af88-4a00-4556-af54-480328ae5e6c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:42.298480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3af88-4a00-4556-af54-480328ae5e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:42.390082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-97a3af88-4a00-4556-af54-480328ae5e6c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:32:42.391244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97a3af88-4a00-4556-af54-480328ae5e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:32:42.503087 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:32:42.660581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b24f8-4db5-4db7-af30-056e1cd9181d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:32:42.664352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e06b24f8-4db5-4db7-af30-056e1cd9181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:32:42.796876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b24f8-4db5-4db7-af30-056e1cd9181d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:32:42.802585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa890d9e-e7c4-4f38-adde-af6e9dadf3dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:32:42.803214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa890d9e-e7c4-4f38-adde-af6e9dadf3dc service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:42.803489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa890d9e-e7c4-4f38-adde-af6e9dadf3dc service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:42.803754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa890d9e-e7c4-4f38-adde-af6e9dadf3dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:42.811919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa890d9e-e7c4-4f38-adde-af6e9dadf3dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:32:42.813495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa890d9e-e7c4-4f38-adde-af6e9dadf3dc service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:32:42.822900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 748/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:32:43.652774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585c2aa-3632-492c-81e2-c5fe82dc1ca8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:43.653372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585c2aa-3632-492c-81e2-c5fe82dc1ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:43.653372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585c2aa-3632-492c-81e2-c5fe82dc1ca8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:43.654679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:32:43.663054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743776-2905-48e6-ac84-c1fb47a6d782 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:43.663767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743776-2905-48e6-ac84-c1fb47a6d782 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:43.664070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743776-2905-48e6-ac84-c1fb47a6d782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:43.689936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-03743776-2905-48e6-ac84-c1fb47a6d782 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:32:43.691185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-03743776-2905-48e6-ac84-c1fb47a6d782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:32:43.700459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:32:43.806733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b6992-943e-47dd-93fc-d726578dc7c9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:43.807129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b6992-943e-47dd-93fc-d726578dc7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:43.807468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b6992-943e-47dd-93fc-d726578dc7c9 None None] PATH_INFO: `/v3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:43.808848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 767/1516] 10.222.0.20 () {54 vars in 925 bytes} [Wed Aug 31 04:32:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:32:43.815922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ae0d-19b0-4958-930d-6a9656844ecd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:43.816204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ae0d-19b0-4958-930d-6a9656844ecd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:43.816468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579ae0d-19b0-4958-930d-6a9656844ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:43.853293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b6992-943e-47dd-93fc-d726578dc7c9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:32:43.869018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b6992-943e-47dd-93fc-d726578dc7c9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:32:43.874742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f115e-4ad8-4a6b-ae99-e30cac65d2b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:32:43.875458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f115e-4ad8-4a6b-ae99-e30cac65d2b9 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:43.875797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f115e-4ad8-4a6b-ae99-e30cac65d2b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:43.876114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f115e-4ad8-4a6b-ae99-e30cac65d2b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:43.880517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6579ae0d-19b0-4958-930d-6a9656844ecd None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:32:43.881705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6579ae0d-19b0-4958-930d-6a9656844ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:32:43.884590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f115e-4ad8-4a6b-ae99-e30cac65d2b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:32:43.886175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f115e-4ad8-4a6b-ae99-e30cac65d2b9 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:32:43.895780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 768/1517] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:32:43.938824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:32:43.959751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 750/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 31 04:32:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:32:53.009891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e58a5-b3a4-4e30-89a8-22614dfdfc38 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:53.010488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e58a5-b3a4-4e30-89a8-22614dfdfc38 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:53.010488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e58a5-b3a4-4e30-89a8-22614dfdfc38 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:53.011743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 769/1519] 10.222.0.42 () {54 vars in 961 bytes} [Wed Aug 31 04:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:32:53.017995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e6f45-b62e-4f89-b2dc-9af75ea8a93f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:53.018697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e6f45-b62e-4f89-b2dc-9af75ea8a93f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:53.018820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e6f45-b62e-4f89-b2dc-9af75ea8a93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:53.047669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-193e6f45-b62e-4f89-b2dc-9af75ea8a93f None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:32:53.048854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-193e6f45-b62e-4f89-b2dc-9af75ea8a93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:32:53.058435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 751/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Aug 31 04:32:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:32:53.066354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e58a5-b3a4-4e30-89a8-22614dfdfc38 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:32:53.081514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e58a5-b3a4-4e30-89a8-22614dfdfc38 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:32:53.087594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01103627-3d95-4449-85fa-75803fec62cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:32:53.088191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103627-3d95-4449-85fa-75803fec62cc service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:53.088479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103627-3d95-4449-85fa-75803fec62cc service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:53.088746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103627-3d95-4449-85fa-75803fec62cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:53.097874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01103627-3d95-4449-85fa-75803fec62cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:32:53.099500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01103627-3d95-4449-85fa-75803fec62cc service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:32:53.109390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 770/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:32:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:32:55.123846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621ddb4-c550-4c67-bae6-404288fc9ace None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:55.124412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621ddb4-c550-4c67-bae6-404288fc9ace None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:55.124412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621ddb4-c550-4c67-bae6-404288fc9ace None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:55.125744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:32:55.131752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52657b38-dd4f-4c67-b4c5-733b482a5b0f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:55.132530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52657b38-dd4f-4c67-b4c5-733b482a5b0f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:55.132694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52657b38-dd4f-4c67-b4c5-733b482a5b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:55.155657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-52657b38-dd4f-4c67-b4c5-733b482a5b0f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:32:55.156854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-52657b38-dd4f-4c67-b4c5-733b482a5b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:32:55.165172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:32:55.321551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621ddb4-c550-4c67-bae6-404288fc9ace None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:32:55.343074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621ddb4-c550-4c67-bae6-404288fc9ace None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:32:55.353334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8f73-8710-40c9-92c3-0a5810bbf298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:32:55.353937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8f73-8710-40c9-92c3-0a5810bbf298 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:55.354207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8f73-8710-40c9-92c3-0a5810bbf298 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:55.354470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8f73-8710-40c9-92c3-0a5810bbf298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:55.363487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8f73-8710-40c9-92c3-0a5810bbf298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:32:55.365069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8f73-8710-40c9-92c3-0a5810bbf298 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:32:55.374587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 753/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:32:56.219318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22ed6-0543-438d-b106-44996577fc39 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:56.219921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22ed6-0543-438d-b106-44996577fc39 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:56.219921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22ed6-0543-438d-b106-44996577fc39 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:56.221864 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:32:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:32:56.230528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6722e8-4fd9-4ba0-8781-90e7eacd83e2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:56.231005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6722e8-4fd9-4ba0-8781-90e7eacd83e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:56.231192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6722e8-4fd9-4ba0-8781-90e7eacd83e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:56.257157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4d6722e8-4fd9-4ba0-8781-90e7eacd83e2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:32:56.258443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d6722e8-4fd9-4ba0-8781-90e7eacd83e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:32:56.268194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:32:56.423563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22ed6-0543-438d-b106-44996577fc39 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:32:56.439756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22ed6-0543-438d-b106-44996577fc39 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:32:56.446194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef060db-083f-4f24-926f-bb4c9f1119f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:32:56.446812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef060db-083f-4f24-926f-bb4c9f1119f2 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:32:56.447088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef060db-083f-4f24-926f-bb4c9f1119f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:32:56.447360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef060db-083f-4f24-926f-bb4c9f1119f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:32:56.455754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef060db-083f-4f24-926f-bb4c9f1119f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:32:56.457341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef060db-083f-4f24-926f-bb4c9f1119f2 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:32:56.467410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 773/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:33:00.143240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43d3ac-5c2b-40ff-bd2e-f54e5696b969 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:00.144126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43d3ac-5c2b-40ff-bd2e-f54e5696b969 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:00.144126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43d3ac-5c2b-40ff-bd2e-f54e5696b969 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:00.146281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:33:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:33:00.152981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015ec97-29ad-44c5-ab83-74ea36182c78 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:00.153446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015ec97-29ad-44c5-ab83-74ea36182c78 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:00.153550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015ec97-29ad-44c5-ab83-74ea36182c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:00.173921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0015ec97-29ad-44c5-ab83-74ea36182c78 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:33:00.175155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0015ec97-29ad-44c5-ab83-74ea36182c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:33:00.184158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:33:00.468413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43d3ac-5c2b-40ff-bd2e-f54e5696b969 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:33:00.471653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a43d3ac-5c2b-40ff-bd2e-f54e5696b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:33:00.554503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43d3ac-5c2b-40ff-bd2e-f54e5696b969 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:33:00.558581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c191150-258e-420e-955c-b9feefcd5ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:33:00.559213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191150-258e-420e-955c-b9feefcd5ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:00.559486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191150-258e-420e-955c-b9feefcd5ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:00.559758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191150-258e-420e-955c-b9feefcd5ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:00.569005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c191150-258e-420e-955c-b9feefcd5ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:33:00.570549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c191150-258e-420e-955c-b9feefcd5ae9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:33:00.580730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 756/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:33:02.471678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84d7a1-36ca-468e-9b20-1a7ba2e25ad3 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:02.472527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84d7a1-36ca-468e-9b20-1a7ba2e25ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:02.472527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84d7a1-36ca-468e-9b20-1a7ba2e25ad3 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:02.474547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:33:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:33:02.480161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5b17-1fda-43e0-9a1e-1de345c885c5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:02.480601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5b17-1fda-43e0-9a1e-1de345c885c5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:02.480704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5b17-1fda-43e0-9a1e-1de345c885c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:02.502996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c16b5b17-1fda-43e0-9a1e-1de345c885c5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:33:02.504192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c16b5b17-1fda-43e0-9a1e-1de345c885c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:33:02.513702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:33:02.794583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84d7a1-36ca-468e-9b20-1a7ba2e25ad3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:33:02.812953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84d7a1-36ca-468e-9b20-1a7ba2e25ad3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:33:02.819317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6fac4-e8b6-46b2-b370-a960e27754c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:33:02.820172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6fac4-e8b6-46b2-b370-a960e27754c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:02.820593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6fac4-e8b6-46b2-b370-a960e27754c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:02.821053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6fac4-e8b6-46b2-b370-a960e27754c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:02.830394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6fac4-e8b6-46b2-b370-a960e27754c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:33:02.831871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6fac4-e8b6-46b2-b370-a960e27754c9 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:33:02.842093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 776/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:33:06.197165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cc9d4-04a2-42c6-96d5-b90efe3a51c5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:06.197746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cc9d4-04a2-42c6-96d5-b90efe3a51c5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:06.197746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cc9d4-04a2-42c6-96d5-b90efe3a51c5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:06.199220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:33:06.205527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f0b44-2132-4640-839e-8e2ded49071f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:06.206254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f0b44-2132-4640-839e-8e2ded49071f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:06.206441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f0b44-2132-4640-839e-8e2ded49071f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:06.218033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:33:06.232043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0f4f0b44-2132-4640-839e-8e2ded49071f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:33:06.233260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0f4f0b44-2132-4640-839e-8e2ded49071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:33:06.241793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:33:06.796123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cc9d4-04a2-42c6-96d5-b90efe3a51c5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:33:06.799319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d96cc9d4-04a2-42c6-96d5-b90efe3a51c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:33:06.891796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cc9d4-04a2-42c6-96d5-b90efe3a51c5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:33:06.895643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba76d9c-34b5-4d30-9796-ea914dc181dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:33:06.896245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba76d9c-34b5-4d30-9796-ea914dc181dd service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:06.896517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba76d9c-34b5-4d30-9796-ea914dc181dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:06.896780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba76d9c-34b5-4d30-9796-ea914dc181dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:06.906159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba76d9c-34b5-4d30-9796-ea914dc181dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:33:06.907617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba76d9c-34b5-4d30-9796-ea914dc181dd service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:33:06.917134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 759/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:33:08.847037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9102d6-b7cd-4f9c-8a82-e87381b822f3 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:08.847623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9102d6-b7cd-4f9c-8a82-e87381b822f3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:08.847623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9102d6-b7cd-4f9c-8a82-e87381b822f3 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:08.848959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:33:08.854467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86ce97-f374-4fe7-b431-db584b94c7af None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:08.854944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86ce97-f374-4fe7-b431-db584b94c7af None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:08.855046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86ce97-f374-4fe7-b431-db584b94c7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:08.876309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3c86ce97-f374-4fe7-b431-db584b94c7af None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:33:08.877491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c86ce97-f374-4fe7-b431-db584b94c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:33:08.885918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:33:09.433004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9102d6-b7cd-4f9c-8a82-e87381b822f3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:33:09.435888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dc9102d6-b7cd-4f9c-8a82-e87381b822f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:33:09.456214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9102d6-b7cd-4f9c-8a82-e87381b822f3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:33:09.461223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0615c09-2173-4ee5-80bf-502723e35450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:33:09.461820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0615c09-2173-4ee5-80bf-502723e35450 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:33:09.462090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0615c09-2173-4ee5-80bf-502723e35450 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:33:09.462353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0615c09-2173-4ee5-80bf-502723e35450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:33:09.473423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0615c09-2173-4ee5-80bf-502723e35450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:33:09.474880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0615c09-2173-4ee5-80bf-502723e35450 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:33:09.484802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 779/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:33:18.871397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:33:19.480507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=8 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:33.150145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a03e54-acd5-4524-b6ae-341dc0804229 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.151271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a03e54-acd5-4524-b6ae-341dc0804229 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.151770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a03e54-acd5-4524-b6ae-341dc0804229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.180135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-80a03e54-acd5-4524-b6ae-341dc0804229 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.181764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-80a03e54-acd5-4524-b6ae-341dc0804229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.201131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 761/1540] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.211832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d99602-561f-4aeb-b917-266c6b06bf52 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.212871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d99602-561f-4aeb-b917-266c6b06bf52 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.213469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d99602-561f-4aeb-b917-266c6b06bf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.238844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-36d99602-561f-4aeb-b917-266c6b06bf52 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.241025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-36d99602-561f-4aeb-b917-266c6b06bf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.253686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 780/1541] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.258962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a03e54-acd5-4524-b6ae-341dc0804229 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.273901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a03e54-acd5-4524-b6ae-341dc0804229 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.282353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.283437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.283937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.284422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.285314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:33.285956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.287370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:34:33.287370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 31 04:34:33.287370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: Aug 31 04:34:33.288709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.297313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fd0b6-801a-4cb5-adbf-692285f6328c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.298002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fd0b6-801a-4cb5-adbf-692285f6328c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.298484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fd0b6-801a-4cb5-adbf-692285f6328c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.322897 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 762/1542] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:33.329863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.335031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6d2fd0b6-801a-4cb5-adbf-692285f6328c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.336895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6d2fd0b6-801a-4cb5-adbf-692285f6328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.347894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f58b2b-1777-4f79-99ee-a53775946cfd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.352828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 781/1543] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.359545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4fb938-0ba4-4f8b-a7d5-10c4454a76fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.360701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb938-0ba4-4f8b-a7d5-10c4454a76fc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.361127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb938-0ba4-4f8b-a7d5-10c4454a76fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.361550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb938-0ba4-4f8b-a7d5-10c4454a76fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.362605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fb938-0ba4-4f8b-a7d5-10c4454a76fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.363004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb180ba-15fd-4119-8c9c-d2a898f9ef1c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.363487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb180ba-15fd-4119-8c9c-d2a898f9ef1c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.363900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb180ba-15fd-4119-8c9c-d2a898f9ef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.365422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4fb938-0ba4-4f8b-a7d5-10c4454a76fc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.399895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-abb180ba-15fd-4119-8c9c-d2a898f9ef1c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.401282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-abb180ba-15fd-4119-8c9c-d2a898f9ef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.415688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 782/1544] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.421607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb180ba-15fd-4119-8c9c-d2a898f9ef1c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.437256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb180ba-15fd-4119-8c9c-d2a898f9ef1c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.445579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 763/1545] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:33.446231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.446894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.447173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.447444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.448109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:33.448408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.448680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070893fc-eb13-4819-83a0-3c7bc4269cff None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.449156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070893fc-eb13-4819-83a0-3c7bc4269cff None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.449673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070893fc-eb13-4819-83a0-3c7bc4269cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.450167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.483811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 783/1546] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:33.484666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.495803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-070893fc-eb13-4819-83a0-3c7bc4269cff None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.497006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-070893fc-eb13-4819-83a0-3c7bc4269cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.503794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29cb83-03d2-45d8-b4f3-a7fc14f0fa1b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.582484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2f1d6-2ee1-4d26-b85e-2d4ce02dc126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.583219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f1d6-2ee1-4d26-b85e-2d4ce02dc126 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.583561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f1d6-2ee1-4d26-b85e-2d4ce02dc126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.583837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2f1d6-2ee1-4d26-b85e-2d4ce02dc126 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.584577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2f1d6-2ee1-4d26-b85e-2d4ce02dc126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.586283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 764/1547] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.586513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2f1d6-2ee1-4d26-b85e-2d4ce02dc126 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.587434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69902153-c42a-44eb-a9cc-6bf002da1468 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.587720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69902153-c42a-44eb-a9cc-6bf002da1468 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.587994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69902153-c42a-44eb-a9cc-6bf002da1468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.619300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-69902153-c42a-44eb-a9cc-6bf002da1468 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.620525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-69902153-c42a-44eb-a9cc-6bf002da1468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.621339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 784/1548] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:33.622665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd820d2-e4ec-4a29-ad4b-d8aba08fe283 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.622997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd820d2-e4ec-4a29-ad4b-d8aba08fe283 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.623315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd820d2-e4ec-4a29-ad4b-d8aba08fe283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.630410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 765/1549] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.631332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69902153-c42a-44eb-a9cc-6bf002da1468 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.646956 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0bd820d2-e4ec-4a29-ad4b-d8aba08fe283 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.648237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0bd820d2-e4ec-4a29-ad4b-d8aba08fe283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.650325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69902153-c42a-44eb-a9cc-6bf002da1468 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.655056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23cb42-e1a4-45f3-9b08-fc99492baa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.655671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cb42-e1a4-45f3-9b08-fc99492baa7e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.655945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cb42-e1a4-45f3-9b08-fc99492baa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.656205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cb42-e1a4-45f3-9b08-fc99492baa7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.656698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 785/1550] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.656889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23cb42-e1a4-45f3-9b08-fc99492baa7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.658753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8346b7-bebe-45d6-bdee-54f70df8253e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.658753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8346b7-bebe-45d6-bdee-54f70df8253e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.658753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8346b7-bebe-45d6-bdee-54f70df8253e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.659116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23cb42-e1a4-45f3-9b08-fc99492baa7e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.680568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4e8346b7-bebe-45d6-bdee-54f70df8253e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.681916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4e8346b7-bebe-45d6-bdee-54f70df8253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.691316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 766/1551] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:33.691789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 786/1552] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.693408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8346b7-bebe-45d6-bdee-54f70df8253e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.696512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1437b5b-cf84-4217-99a3-91449671d275 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.696857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1437b5b-cf84-4217-99a3-91449671d275 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.697303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1437b5b-cf84-4217-99a3-91449671d275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.704733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8346b7-bebe-45d6-bdee-54f70df8253e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.721711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e1437b5b-cf84-4217-99a3-91449671d275 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.722965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e1437b5b-cf84-4217-99a3-91449671d275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.771384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3338a61-a703-499c-b0d5-3daddb0af556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.772358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3338a61-a703-499c-b0d5-3daddb0af556 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.773331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3338a61-a703-499c-b0d5-3daddb0af556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.773617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3338a61-a703-499c-b0d5-3daddb0af556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.774282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3338a61-a703-499c-b0d5-3daddb0af556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.776028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3338a61-a703-499c-b0d5-3daddb0af556 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.786824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 767/1553] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.790666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 787/1554] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:33.790873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a6511-8e80-4a81-9334-c27c96593b5e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.790873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a6511-8e80-4a81-9334-c27c96593b5e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.790873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a6511-8e80-4a81-9334-c27c96593b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.791156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b990c2-b1fb-445e-9e42-724add21c4d3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.791156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b990c2-b1fb-445e-9e42-724add21c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.791156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b990c2-b1fb-445e-9e42-724add21c4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.809841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-86b990c2-b1fb-445e-9e42-724add21c4d3 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.810612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1e2a6511-8e80-4a81-9334-c27c96593b5e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.811083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-86b990c2-b1fb-445e-9e42-724add21c4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.811799 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1e2a6511-8e80-4a81-9334-c27c96593b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.820203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 788/1555] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.821584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedc37e-cf0b-4f14-8b0d-8b7f369445c4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.821584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedc37e-cf0b-4f14-8b0d-8b7f369445c4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.826815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 768/1556] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.826913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedc37e-cf0b-4f14-8b0d-8b7f369445c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.826997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68329c09-5131-40b4-a4ac-d88c8c82db74 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.826997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68329c09-5131-40b4-a4ac-d88c8c82db74 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.826997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68329c09-5131-40b4-a4ac-d88c8c82db74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.846235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3bedc37e-cf0b-4f14-8b0d-8b7f369445c4 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.847469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3bedc37e-cf0b-4f14-8b0d-8b7f369445c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.850773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-68329c09-5131-40b4-a4ac-d88c8c82db74 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.852046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-68329c09-5131-40b4-a4ac-d88c8c82db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.856212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 789/1557] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.857010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedc37e-cf0b-4f14-8b0d-8b7f369445c4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.867255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedc37e-cf0b-4f14-8b0d-8b7f369445c4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.868020 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 769/1558] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.870181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae0fd-2db3-4bfa-b339-bc94b6aee7c0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.870462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae0fd-2db3-4bfa-b339-bc94b6aee7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.870699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ae0fd-2db3-4bfa-b339-bc94b6aee7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.875224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4f300-78d4-40a3-95b6-5b254509fe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.875926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4f300-78d4-40a3-95b6-5b254509fe1d admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.876203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4f300-78d4-40a3-95b6-5b254509fe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.876477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4f300-78d4-40a3-95b6-5b254509fe1d admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.877112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4f300-78d4-40a3-95b6-5b254509fe1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.879096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4f300-78d4-40a3-95b6-5b254509fe1d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.894324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b49ae0fd-2db3-4bfa-b339-bc94b6aee7c0 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.895616 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b49ae0fd-2db3-4bfa-b339-bc94b6aee7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.905486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 790/1559] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:33.905486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4f300-78d4-40a3-95b6-5b254509fe1d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.909916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 770/1560] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.911350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ae0fd-2db3-4bfa-b339-bc94b6aee7c0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.916583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4f300-78d4-40a3-95b6-5b254509fe1d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.917030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:33.922403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ae0fd-2db3-4bfa-b339-bc94b6aee7c0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.923932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.924545 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.924807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.925087 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.925736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:33.926036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.927818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.929595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e782d18-7eea-440b-9b43-b3d18d8ee2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.930252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782d18-7eea-440b-9b43-b3d18d8ee2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.930528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782d18-7eea-440b-9b43-b3d18d8ee2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.930823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782d18-7eea-440b-9b43-b3d18d8ee2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.931631 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e782d18-7eea-440b-9b43-b3d18d8ee2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.932057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:33.933305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e782d18-7eea-440b-9b43-b3d18d8ee2e7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.952395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 791/1561] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:33.952637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 771/1562] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:33.952761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:33.952880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1983-0f4a-4fb6-b0ff-e932679e1bbf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.952880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1983-0f4a-4fb6-b0ff-e932679e1bbf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.952880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1983-0f4a-4fb6-b0ff-e932679e1bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.957429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9d9a0-ba47-42d2-9b10-5dc8e6aab73d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:33.963466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:33.964073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.964350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.964610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.965278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:33.965579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:33.967229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:33.975415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-350f1983-0f4a-4fb6-b0ff-e932679e1bbf None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:33.976628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-350f1983-0f4a-4fb6-b0ff-e932679e1bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:33.981454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 792/1563] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:33.986691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 772/1564] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:33.987925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea244a-d5fd-454d-b972-e80d8ff1d565 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:33.988313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea244a-d5fd-454d-b972-e80d8ff1d565 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:33.989010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea244a-d5fd-454d-b972-e80d8ff1d565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:33.990582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.001496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae5d5c-2fe8-44eb-8c33-4f465e90aebc admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.006438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.007095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.007374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.007654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.008309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:34.008613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.010235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.015516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-68ea244a-d5fd-454d-b972-e80d8ff1d565 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:34.016736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-68ea244a-d5fd-454d-b972-e80d8ff1d565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:34.032341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 793/1565] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:34.032341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.032647 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 773/1566] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:34.032647 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea244a-d5fd-454d-b972-e80d8ff1d565 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.036306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea244a-d5fd-454d-b972-e80d8ff1d565 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.041444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.042081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.042357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.042624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.043320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:34.043619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.044036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36819c58-daa2-49bd-9a14-68b3cd0d106d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.045228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.049317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d32a-d412-483a-8db6-327add0685e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.050033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d32a-d412-483a-8db6-327add0685e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.050345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d32a-d412-483a-8db6-327add0685e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.050629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d32a-d412-483a-8db6-327add0685e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.051296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d32a-d412-483a-8db6-327add0685e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.053442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d32a-d412-483a-8db6-327add0685e4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.058074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 774/1567] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:34.058921 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.064820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 794/1568] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:34.066798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d32a-d412-483a-8db6-327add0685e4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.079200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0383f28-5db0-4cc1-8dcd-31ac762c56f2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.080356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d32a-d412-483a-8db6-327add0685e4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.084346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80817c-6f89-4973-b001-b16d57d29bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.085114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80817c-6f89-4973-b001-b16d57d29bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.085277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.085385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80817c-6f89-4973-b001-b16d57d29bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.085640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80817c-6f89-4973-b001-b16d57d29bbe admin admin] PATH_INFO: `/v3/projects/2967ca7d73584b45baa158f5994f64df/users/0f863757bba840ea85db119179b9fc7c/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.085897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.086179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.086448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.087110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.088340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:34:34.088340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Aug 31 04:34:34.088340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: Aug 31 04:34:34.090452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.102245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80817c-6f89-4973-b001-b16d57d29bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2967ca7d73584b45baa158f5994f64df, user_id=0f863757bba840ea85db119179b9fc7c, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.104880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80817c-6f89-4973-b001-b16d57d29bbe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.120792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 775/1569] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:34 2022] PUT /identity/v3/projects/2967ca7d73584b45baa158f5994f64df/users/0f863757bba840ea85db119179b9fc7c/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:34.121636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80817c-6f89-4973-b001-b16d57d29bbe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.126945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 795/1570] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.126945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.193817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f8622-300e-4715-a045-0788df1ddf36 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.195874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80817c-6f89-4973-b001-b16d57d29bbe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.198740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.199425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.199721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.200051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.200655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:34.200956 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.201373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84386334-d919-4fb7-905a-39fd2330366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.201996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84386334-d919-4fb7-905a-39fd2330366c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.202269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84386334-d919-4fb7-905a-39fd2330366c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.202608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.202743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84386334-d919-4fb7-905a-39fd2330366c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.203842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84386334-d919-4fb7-905a-39fd2330366c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.206554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84386334-d919-4fb7-905a-39fd2330366c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.224532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 796/1571] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:34.224532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.235333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691965a9-7581-46e7-a2fd-20ce87f14db2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.240384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.241007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.241285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.241554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.242003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 776/1572] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/projects => generated 391 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.242222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:34.242502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.242947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84386334-d919-4fb7-905a-39fd2330366c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.244150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.258454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 797/1573] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:34.259701 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.270740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832271e-a4cf-4e54-9d08-66f4aba97777 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.313365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84386334-d919-4fb7-905a-39fd2330366c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.318804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a31d1-9613-44fa-9155-1aa8ffc049ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.319428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a31d1-9613-44fa-9155-1aa8ffc049ba admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.319709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a31d1-9613-44fa-9155-1aa8ffc049ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.319976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a31d1-9613-44fa-9155-1aa8ffc049ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.320647 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a31d1-9613-44fa-9155-1aa8ffc049ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.322648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a31d1-9613-44fa-9155-1aa8ffc049ba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.333768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eb988-4b82-4e68-8fb7-4c8054d5e4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.334392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eb988-4b82-4e68-8fb7-4c8054d5e4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.334679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eb988-4b82-4e68-8fb7-4c8054d5e4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.334984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eb988-4b82-4e68-8fb7-4c8054d5e4a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.335638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7eb988-4b82-4e68-8fb7-4c8054d5e4a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.337290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7eb988-4b82-4e68-8fb7-4c8054d5e4a1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.348787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 798/1574] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:34.350341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eb988-4b82-4e68-8fb7-4c8054d5e4a1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.354245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 777/1575] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.355526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4f4e0-5d5e-4472-ae56-9b1bd12dbfb6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.355824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4f4e0-5d5e-4472-ae56-9b1bd12dbfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.356100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4f4e0-5d5e-4472-ae56-9b1bd12dbfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.362788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eb988-4b82-4e68-8fb7-4c8054d5e4a1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.374702 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:34.375604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2df4f4e0-5d5e-4472-ae56-9b1bd12dbfb6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:34.376985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2df4f4e0-5d5e-4472-ae56-9b1bd12dbfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:34.384307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:34.429569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c9a74-39b5-4d32-b1c2-edebbf67232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.430190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9a74-39b5-4d32-b1c2-edebbf67232a admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.430467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9a74-39b5-4d32-b1c2-edebbf67232a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.430764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9a74-39b5-4d32-b1c2-edebbf67232a admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.431416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c9a74-39b5-4d32-b1c2-edebbf67232a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.433361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c9a74-39b5-4d32-b1c2-edebbf67232a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.448609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 778/1576] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:34.449632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4f4e0-5d5e-4472-ae56-9b1bd12dbfb6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.461400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4f4e0-5d5e-4472-ae56-9b1bd12dbfb6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.473247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 799/1577] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.473448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.473448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.473727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c9a74-39b5-4d32-b1c2-edebbf67232a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.473885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.473885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.473885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.473885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:34:34.473885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 31 04:34:34.473885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: Aug 31 04:34:34.473885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.498815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 779/1578] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.499730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.510145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c33600-d47c-48b3-9b3d-9f9587dcd56a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.542322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c9a74-39b5-4d32-b1c2-edebbf67232a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.547197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1234d5-6e59-47be-be3d-3031b02c6529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.547839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1234d5-6e59-47be-be3d-3031b02c6529 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.548097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1234d5-6e59-47be-be3d-3031b02c6529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.548363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1234d5-6e59-47be-be3d-3031b02c6529 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.548992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1234d5-6e59-47be-be3d-3031b02c6529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.550958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1234d5-6e59-47be-be3d-3031b02c6529 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.571714 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbd778-1f0d-40e1-88b8-770fa29607fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.572336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd778-1f0d-40e1-88b8-770fa29607fb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.572613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd778-1f0d-40e1-88b8-770fa29607fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.572884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd778-1f0d-40e1-88b8-770fa29607fb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.573518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbd778-1f0d-40e1-88b8-770fa29607fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.575495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbd778-1f0d-40e1-88b8-770fa29607fb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.578824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 800/1579] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.583197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1234d5-6e59-47be-be3d-3031b02c6529 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.598145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 780/1580] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.599313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbd778-1f0d-40e1-88b8-770fa29607fb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.608793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbd778-1f0d-40e1-88b8-770fa29607fb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.653076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1234d5-6e59-47be-be3d-3031b02c6529 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.658308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7c544-11bb-4b42-b753-59fca05a622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.658738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7c544-11bb-4b42-b753-59fca05a622f admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.659033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7c544-11bb-4b42-b753-59fca05a622f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.659359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7c544-11bb-4b42-b753-59fca05a622f admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.659924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7c544-11bb-4b42-b753-59fca05a622f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.661913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7c544-11bb-4b42-b753-59fca05a622f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.665336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cd3cf-ca7c-4806-a8ae-23986de1c4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.666008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd3cf-ca7c-4806-a8ae-23986de1c4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.666293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd3cf-ca7c-4806-a8ae-23986de1c4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.666571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd3cf-ca7c-4806-a8ae-23986de1c4c3 admin admin] PATH_INFO: `/v3/projects/3bd54eb751f144b29b274e6a96dedcb2/users/13ac62bdb9bc4c7f94dad08f810559e9/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.683129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cd3cf-ca7c-4806-a8ae-23986de1c4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd54eb751f144b29b274e6a96dedcb2, user_id=13ac62bdb9bc4c7f94dad08f810559e9, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.686457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cd3cf-ca7c-4806-a8ae-23986de1c4c3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.690202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 801/1581] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.691221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7c544-11bb-4b42-b753-59fca05a622f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.700995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 781/1582] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:34 2022] PUT /identity/v3/projects/3bd54eb751f144b29b274e6a96dedcb2/users/13ac62bdb9bc4c7f94dad08f810559e9/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:34.701830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cd3cf-ca7c-4806-a8ae-23986de1c4c3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.702038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:34.711422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:34.757012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7c544-11bb-4b42-b753-59fca05a622f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.759813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cd3cf-ca7c-4806-a8ae-23986de1c4c3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.764381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a01a6-36d7-4391-99c3-5482e7cc0c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.765016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a01a6-36d7-4391-99c3-5482e7cc0c83 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.765289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a01a6-36d7-4391-99c3-5482e7cc0c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.765554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a01a6-36d7-4391-99c3-5482e7cc0c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.813617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92020c7-4d40-494f-8aac-2c2483f1eac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.814236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92020c7-4d40-494f-8aac-2c2483f1eac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.814513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92020c7-4d40-494f-8aac-2c2483f1eac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.814825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92020c7-4d40-494f-8aac-2c2483f1eac7 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.815462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92020c7-4d40-494f-8aac-2c2483f1eac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.817693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92020c7-4d40-494f-8aac-2c2483f1eac7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.825456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a01a6-36d7-4391-99c3-5482e7cc0c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.826922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a01a6-36d7-4391-99c3-5482e7cc0c83 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.837786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 782/1583] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:34.838599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a01a6-36d7-4391-99c3-5482e7cc0c83 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.841138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 802/1584] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.842166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92020c7-4d40-494f-8aac-2c2483f1eac7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.848836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a01a6-36d7-4391-99c3-5482e7cc0c83 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.855030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92020c7-4d40-494f-8aac-2c2483f1eac7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.856089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a78ff-9389-42e5-83d9-3679939845b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.856713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a78ff-9389-42e5-83d9-3679939845b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.856993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a78ff-9389-42e5-83d9-3679939845b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.857268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a78ff-9389-42e5-83d9-3679939845b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.857909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a78ff-9389-42e5-83d9-3679939845b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.859784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee4bf7-5024-4062-bb1e-bfd4dfb16806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.859923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a78ff-9389-42e5-83d9-3679939845b6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.860334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee4bf7-5024-4062-bb1e-bfd4dfb16806 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.860611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee4bf7-5024-4062-bb1e-bfd4dfb16806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.860879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee4bf7-5024-4062-bb1e-bfd4dfb16806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.861620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee4bf7-5024-4062-bb1e-bfd4dfb16806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.863447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee4bf7-5024-4062-bb1e-bfd4dfb16806 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.876874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 803/1585] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:34.877807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee4bf7-5024-4062-bb1e-bfd4dfb16806 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.888981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee4bf7-5024-4062-bb1e-bfd4dfb16806 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.892192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 783/1586] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/users => generated 286 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.892192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a78ff-9389-42e5-83d9-3679939845b6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.894274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431a4d8-4b5b-4693-80d6-2bf5e4672eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.894934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431a4d8-4b5b-4693-80d6-2bf5e4672eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.895350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431a4d8-4b5b-4693-80d6-2bf5e4672eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.895740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431a4d8-4b5b-4693-80d6-2bf5e4672eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.896521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431a4d8-4b5b-4693-80d6-2bf5e4672eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.898447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431a4d8-4b5b-4693-80d6-2bf5e4672eaf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.900103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a78ff-9389-42e5-83d9-3679939845b6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.904869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97efe4-9265-4c94-adc9-3dd5ddbb057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.905473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97efe4-9265-4c94-adc9-3dd5ddbb057e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.905755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97efe4-9265-4c94-adc9-3dd5ddbb057e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.906030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97efe4-9265-4c94-adc9-3dd5ddbb057e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.906681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97efe4-9265-4c94-adc9-3dd5ddbb057e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.908382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97efe4-9265-4c94-adc9-3dd5ddbb057e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.921924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 784/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:34.921924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97efe4-9265-4c94-adc9-3dd5ddbb057e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.929910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 804/1588] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/users => generated 294 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.931469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3323c3-0ea2-4853-b224-ce55a7b50e44 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.931758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3323c3-0ea2-4853-b224-ce55a7b50e44 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.932057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3323c3-0ea2-4853-b224-ce55a7b50e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.934704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97efe4-9265-4c94-adc9-3dd5ddbb057e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.940128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71b2f5-a8af-46a0-adf5-e9f295794a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.940798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71b2f5-a8af-46a0-adf5-e9f295794a69 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.941082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71b2f5-a8af-46a0-adf5-e9f295794a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.941352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71b2f5-a8af-46a0-adf5-e9f295794a69 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.942460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71b2f5-a8af-46a0-adf5-e9f295794a69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.944390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71b2f5-a8af-46a0-adf5-e9f295794a69 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:34.956428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8b3323c3-0ea2-4853-b224-ce55a7b50e44 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:34.957880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8b3323c3-0ea2-4853-b224-ce55a7b50e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:34.971196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 805/1590] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:34.971196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3323c3-0ea2-4853-b224-ce55a7b50e44 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.972837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 785/1590] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:34 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:34.974338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71b2f5-a8af-46a0-adf5-e9f295794a69 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:34.989955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71b2f5-a8af-46a0-adf5-e9f295794a69 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.990472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3323c3-0ea2-4853-b224-ce55a7b50e44 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:34.996015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af96eb-77aa-41de-9f6a-a0ffaf048c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.996644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af96eb-77aa-41de-9f6a-a0ffaf048c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.996774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e92282-8ff3-4a8c-b2fe-fc95db7043cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:34.996994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af96eb-77aa-41de-9f6a-a0ffaf048c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.997475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af96eb-77aa-41de-9f6a-a0ffaf048c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.997667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e92282-8ff3-4a8c-b2fe-fc95db7043cf admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:34.997667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e92282-8ff3-4a8c-b2fe-fc95db7043cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:34.997950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e92282-8ff3-4a8c-b2fe-fc95db7043cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:34.998061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af96eb-77aa-41de-9f6a-a0ffaf048c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:34.998595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e92282-8ff3-4a8c-b2fe-fc95db7043cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.000138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e92282-8ff3-4a8c-b2fe-fc95db7043cf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.000259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af96eb-77aa-41de-9f6a-a0ffaf048c86 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.015181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 786/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.015554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 806/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.016182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e92282-8ff3-4a8c-b2fe-fc95db7043cf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.017113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af96eb-77aa-41de-9f6a-a0ffaf048c86 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.029064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e92282-8ff3-4a8c-b2fe-fc95db7043cf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.029362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af96eb-77aa-41de-9f6a-a0ffaf048c86 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.034357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde6e97-ba91-4a59-92d1-a462316ada4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.034702 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465f73-d34c-4542-b17a-3176521d934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.034999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde6e97-ba91-4a59-92d1-a462316ada4d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.035341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde6e97-ba91-4a59-92d1-a462316ada4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.035440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465f73-d34c-4542-b17a-3176521d934e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.035675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465f73-d34c-4542-b17a-3176521d934e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.035872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde6e97-ba91-4a59-92d1-a462316ada4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.036037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465f73-d34c-4542-b17a-3176521d934e admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.036424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde6e97-ba91-4a59-92d1-a462316ada4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.036847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06465f73-d34c-4542-b17a-3176521d934e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.038237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06465f73-d34c-4542-b17a-3176521d934e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.038340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde6e97-ba91-4a59-92d1-a462316ada4d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.054734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 807/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.055786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465f73-d34c-4542-b17a-3176521d934e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.068366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 787/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.068366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde6e97-ba91-4a59-92d1-a462316ada4d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.072818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465f73-d34c-4542-b17a-3176521d934e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.077661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.078383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.078691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.079066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.079800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.080720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde6e97-ba91-4a59-92d1-a462316ada4d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.081546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.085902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a307d-c271-463f-859f-364c4e936318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.086509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a307d-c271-463f-859f-364c4e936318 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.086852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a307d-c271-463f-859f-364c4e936318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.087134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a307d-c271-463f-859f-364c4e936318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.088443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a307d-c271-463f-859f-364c4e936318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.090102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a307d-c271-463f-859f-364c4e936318 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.095193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 808/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.098660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.100449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:35.106149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 788/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.106149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a307d-c271-463f-859f-364c4e936318 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.117077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a307d-c271-463f-859f-364c4e936318 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.117786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0480740-21be-45f9-8938-c3cd1fd8dfe9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.121873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86e375-96ca-416e-91be-20a09ce54fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.122345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08331b-ee28-42bc-8107-8eb012a019ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.122677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86e375-96ca-416e-91be-20a09ce54fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.123119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86e375-96ca-416e-91be-20a09ce54fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.123363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86e375-96ca-416e-91be-20a09ce54fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.123503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08331b-ee28-42bc-8107-8eb012a019ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.123684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08331b-ee28-42bc-8107-8eb012a019ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.123972 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08331b-ee28-42bc-8107-8eb012a019ab admin admin] PATH_INFO: `/v3/projects/94afda48b8e545dfaeac33a66f592fe2/users/718418f954f94dad93eeb725e345c850/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.137425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86e375-96ca-416e-91be-20a09ce54fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.139082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86e375-96ca-416e-91be-20a09ce54fbd service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.143002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08331b-ee28-42bc-8107-8eb012a019ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=94afda48b8e545dfaeac33a66f592fe2, user_id=718418f954f94dad93eeb725e345c850, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.145893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08331b-ee28-42bc-8107-8eb012a019ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.149772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 809/1597] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:35.150514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86e375-96ca-416e-91be-20a09ce54fbd service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.161521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86e375-96ca-416e-91be-20a09ce54fbd service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.163955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 789/1598] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:35 2022] PUT /identity/v3/projects/94afda48b8e545dfaeac33a66f592fe2/users/718418f954f94dad93eeb725e345c850/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:35.164927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08331b-ee28-42bc-8107-8eb012a019ab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.183065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:35.203014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:35.260038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cda1b-a868-492d-ae94-39a5372c390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.261944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cda1b-a868-492d-ae94-39a5372c390d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.262514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cda1b-a868-492d-ae94-39a5372c390d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.262974 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cda1b-a868-492d-ae94-39a5372c390d admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.263821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cda1b-a868-492d-ae94-39a5372c390d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.265951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cda1b-a868-492d-ae94-39a5372c390d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.279068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 810/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.279764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08331b-ee28-42bc-8107-8eb012a019ab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.280929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cda1b-a868-492d-ae94-39a5372c390d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.289263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148cd3c-18bc-4159-ba59-9ae8bb2c3773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.291358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148cd3c-18bc-4159-ba59-9ae8bb2c3773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.291668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148cd3c-18bc-4159-ba59-9ae8bb2c3773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.291998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148cd3c-18bc-4159-ba59-9ae8bb2c3773 admin admin] PATH_INFO: `/v3/projects/c02e60929267489cb181831f1ac578e4/users/863e457bd6fd4bff82f552e63ca1034c/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.296149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cda1b-a868-492d-ae94-39a5372c390d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.302079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b4f6b-6777-4404-8195-9a9820e1332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.302706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b4f6b-6777-4404-8195-9a9820e1332b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.303055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b4f6b-6777-4404-8195-9a9820e1332b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.303325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b4f6b-6777-4404-8195-9a9820e1332b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.304104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b4f6b-6777-4404-8195-9a9820e1332b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.306230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b4f6b-6777-4404-8195-9a9820e1332b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.317711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 811/1600] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.318552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b4f6b-6777-4404-8195-9a9820e1332b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.326903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148cd3c-18bc-4159-ba59-9ae8bb2c3773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02e60929267489cb181831f1ac578e4, user_id=863e457bd6fd4bff82f552e63ca1034c, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.330005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148cd3c-18bc-4159-ba59-9ae8bb2c3773 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.330258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b4f6b-6777-4404-8195-9a9820e1332b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.335783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897e992-e2fd-418e-8bad-e4563d22c193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.336436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897e992-e2fd-418e-8bad-e4563d22c193 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.336737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897e992-e2fd-418e-8bad-e4563d22c193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.336960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897e992-e2fd-418e-8bad-e4563d22c193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.337952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897e992-e2fd-418e-8bad-e4563d22c193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.339916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897e992-e2fd-418e-8bad-e4563d22c193 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.351082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 790/1601] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:35 2022] PUT /identity/v3/projects/c02e60929267489cb181831f1ac578e4/users/863e457bd6fd4bff82f552e63ca1034c/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:35.352246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148cd3c-18bc-4159-ba59-9ae8bb2c3773 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.352787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 812/1602] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.357358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd86db-f8cb-4760-9c6f-ea8e51082a9c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.357779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd86db-f8cb-4760-9c6f-ea8e51082a9c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.358225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd86db-f8cb-4760-9c6f-ea8e51082a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.380833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-97bd86db-f8cb-4760-9c6f-ea8e51082a9c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:35.382162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97bd86db-f8cb-4760-9c6f-ea8e51082a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:35.431885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148cd3c-18bc-4159-ba59-9ae8bb2c3773 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.437958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe546-959d-414b-8af2-7f0719f233c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.438583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe546-959d-414b-8af2-7f0719f233c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.438901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe546-959d-414b-8af2-7f0719f233c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.439222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe546-959d-414b-8af2-7f0719f233c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.439888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbe546-959d-414b-8af2-7f0719f233c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.441637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbe546-959d-414b-8af2-7f0719f233c4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.451884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 813/1603] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:35.451884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd86db-f8cb-4760-9c6f-ea8e51082a9c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.454800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 791/1604] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:35.455647 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe546-959d-414b-8af2-7f0719f233c4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.463730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd86db-f8cb-4760-9c6f-ea8e51082a9c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.469658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7141e-75da-4314-b653-82f66d8a35b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.470256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe546-959d-414b-8af2-7f0719f233c4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.470386 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7141e-75da-4314-b653-82f66d8a35b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.470589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7141e-75da-4314-b653-82f66d8a35b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.471235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7141e-75da-4314-b653-82f66d8a35b8 admin admin] PATH_INFO: `/v3/projects/25a70b14a07b44a68f0a998219eb5587/users/eaf82e5e73d04955962c300d055376e6/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.476466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4937bf0-421a-44ae-9501-7990ce2cc4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.477222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4937bf0-421a-44ae-9501-7990ce2cc4f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.477417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4937bf0-421a-44ae-9501-7990ce2cc4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.477678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4937bf0-421a-44ae-9501-7990ce2cc4f6 admin admin] PATH_INFO: `/v3/projects/3fd095529d224079ac9f991ca414526b/users/23dd1dbd89174c8092cb59e9b021967c/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.493923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7141e-75da-4314-b653-82f66d8a35b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a70b14a07b44a68f0a998219eb5587, user_id=eaf82e5e73d04955962c300d055376e6, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.495249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-a8d7141e-75da-4314-b653-82f66d8a35b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:34:35.495249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Aug 31 04:34:35.495249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: Aug 31 04:34:35.496268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4937bf0-421a-44ae-9501-7990ce2cc4f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd095529d224079ac9f991ca414526b, user_id=23dd1dbd89174c8092cb59e9b021967c, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.497656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7141e-75da-4314-b653-82f66d8a35b8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.500906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4937bf0-421a-44ae-9501-7990ce2cc4f6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.515530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 814/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:35 2022] PUT /identity/v3/projects/25a70b14a07b44a68f0a998219eb5587/users/eaf82e5e73d04955962c300d055376e6/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:35.517083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cf3fa-7ffc-43e3-a41d-8a2a5c422d24 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.517490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cf3fa-7ffc-43e3-a41d-8a2a5c422d24 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.517871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cf3fa-7ffc-43e3-a41d-8a2a5c422d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.544384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bc1cf3fa-7ffc-43e3-a41d-8a2a5c422d24 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:35.545852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bc1cf3fa-7ffc-43e3-a41d-8a2a5c422d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:35.546536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 792/1606] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:35 2022] PUT /identity/v3/projects/3fd095529d224079ac9f991ca414526b/users/23dd1dbd89174c8092cb59e9b021967c/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:35.548172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4937bf0-421a-44ae-9501-7990ce2cc4f6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.658392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4937bf0-421a-44ae-9501-7990ce2cc4f6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.666544 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd593f-c866-43bd-8888-f69a5082e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.668064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd593f-c866-43bd-8888-f69a5082e0de admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.668465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd593f-c866-43bd-8888-f69a5082e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.668746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd593f-c866-43bd-8888-f69a5082e0de admin admin] PATH_INFO: `/v3/projects/c6907a65652e4be9ab958617329e7484/users/05d7ba531c534b76a1b214d6501f49b2/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.674258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 815/1607] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:35.676744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cf3fa-7ffc-43e3-a41d-8a2a5c422d24 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.684164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:35.689436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd593f-c866-43bd-8888-f69a5082e0de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6907a65652e4be9ab958617329e7484, user_id=05d7ba531c534b76a1b214d6501f49b2, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.693243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd593f-c866-43bd-8888-f69a5082e0de admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.693883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cf3fa-7ffc-43e3-a41d-8a2a5c422d24 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.701279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab91f7-fdec-495d-b4f9-9c2b51847f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.702278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab91f7-fdec-495d-b4f9-9c2b51847f4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.703283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab91f7-fdec-495d-b4f9-9c2b51847f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.703283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab91f7-fdec-495d-b4f9-9c2b51847f4a admin admin] PATH_INFO: `/v3/projects/9dbade89d75a4cd39a77873b1190e072/users/6d9c94625d8f4237b3ea8667ebc22259/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.719960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:35.725339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab91f7-fdec-495d-b4f9-9c2b51847f4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dbade89d75a4cd39a77873b1190e072, user_id=6d9c94625d8f4237b3ea8667ebc22259, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:35.728568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab91f7-fdec-495d-b4f9-9c2b51847f4a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:35.740654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 793/1608] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:35 2022] PUT /identity/v3/projects/c6907a65652e4be9ab958617329e7484/users/05d7ba531c534b76a1b214d6501f49b2/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:35.744351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd593f-c866-43bd-8888-f69a5082e0de admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:35.760359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 816/1609] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:35 2022] PUT /identity/v3/projects/9dbade89d75a4cd39a77873b1190e072/users/6d9c94625d8f4237b3ea8667ebc22259/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:35.761802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e058243-408f-4b47-9cb1-0c9c42b66616 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.762085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e058243-408f-4b47-9cb1-0c9c42b66616 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.762356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e058243-408f-4b47-9cb1-0c9c42b66616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.810564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0e058243-408f-4b47-9cb1-0c9c42b66616 None None] MFA Rules not processed for user `eaf82e5e73d04955962c300d055376e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:35.811885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0e058243-408f-4b47-9cb1-0c9c42b66616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:35.840452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd593f-c866-43bd-8888-f69a5082e0de admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:35.897259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 817/1610] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:35.898590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cdbb2-726f-49aa-abdb-23a1f963ac2e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.898940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cdbb2-726f-49aa-abdb-23a1f963ac2e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.899221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cdbb2-726f-49aa-abdb-23a1f963ac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.923032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52873575-226d-4cfe-ab1c-8bf174158731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:35.924315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52873575-226d-4cfe-ab1c-8bf174158731 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:35.924793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52873575-226d-4cfe-ab1c-8bf174158731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:35.925351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52873575-226d-4cfe-ab1c-8bf174158731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:35.936565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-099cdbb2-726f-49aa-abdb-23a1f963ac2e None None] MFA Rules not processed for user `23dd1dbd89174c8092cb59e9b021967c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:35.937855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-099cdbb2-726f-49aa-abdb-23a1f963ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:36.007964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52873575-226d-4cfe-ab1c-8bf174158731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.010389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52873575-226d-4cfe-ab1c-8bf174158731 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.023054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 818/1611] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:36.023653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 794/1612] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:36.023928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cdbb2-726f-49aa-abdb-23a1f963ac2e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.025341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743961-9341-4281-817d-15d8283d7c6e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.026086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743961-9341-4281-817d-15d8283d7c6e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.026745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743961-9341-4281-817d-15d8283d7c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.035554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cdbb2-726f-49aa-abdb-23a1f963ac2e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.040194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95cae7-cf78-4b67-9d88-ea24f7f02fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.040821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95cae7-cf78-4b67-9d88-ea24f7f02fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.041146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95cae7-cf78-4b67-9d88-ea24f7f02fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.041416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95cae7-cf78-4b67-9d88-ea24f7f02fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.054692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95cae7-cf78-4b67-9d88-ea24f7f02fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.057330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95cae7-cf78-4b67-9d88-ea24f7f02fb0 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.071467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e9743961-9341-4281-817d-15d8283d7c6e None None] MFA Rules not processed for user `05d7ba531c534b76a1b214d6501f49b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:36.072402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 819/1613] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:36.073164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e9743961-9341-4281-817d-15d8283d7c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:36.074106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416f12e-636b-4c08-a79b-4c0e8a2ca0b8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.074751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416f12e-636b-4c08-a79b-4c0e8a2ca0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.075226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416f12e-636b-4c08-a79b-4c0e8a2ca0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.116913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2416f12e-636b-4c08-a79b-4c0e8a2ca0b8 None None] MFA Rules not processed for user `6d9c94625d8f4237b3ea8667ebc22259`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:36.118937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2416f12e-636b-4c08-a79b-4c0e8a2ca0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:36.155248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:36.190106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:36.191889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 795/1614] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:36.193379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743961-9341-4281-817d-15d8283d7c6e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.205649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743961-9341-4281-817d-15d8283d7c6e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.211034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d3547-bc47-4941-892d-cd8b4d2c6abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.212279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d3547-bc47-4941-892d-cd8b4d2c6abe admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.212564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d3547-bc47-4941-892d-cd8b4d2c6abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.212882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d3547-bc47-4941-892d-cd8b4d2c6abe admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.213697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d3547-bc47-4941-892d-cd8b4d2c6abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.215800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d3547-bc47-4941-892d-cd8b4d2c6abe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.226112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 820/1615] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:36.227406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416f12e-636b-4c08-a79b-4c0e8a2ca0b8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.244795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416f12e-636b-4c08-a79b-4c0e8a2ca0b8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.249690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb8287-9d01-44e9-b18e-35d56117e505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.251157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb8287-9d01-44e9-b18e-35d56117e505 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.251473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb8287-9d01-44e9-b18e-35d56117e505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.251871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb8287-9d01-44e9-b18e-35d56117e505 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.252913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb8287-9d01-44e9-b18e-35d56117e505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.256362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb8287-9d01-44e9-b18e-35d56117e505 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.257243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 796/1616] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:36.258575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d3547-bc47-4941-892d-cd8b4d2c6abe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.300385 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 821/1617] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:36.302375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb8287-9d01-44e9-b18e-35d56117e505 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.377832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d3547-bc47-4941-892d-cd8b4d2c6abe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.398054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb8287-9d01-44e9-b18e-35d56117e505 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.404553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff427ff-b3dd-4f9b-a73e-615958e86cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.405746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff427ff-b3dd-4f9b-a73e-615958e86cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.406195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff427ff-b3dd-4f9b-a73e-615958e86cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.406617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff427ff-b3dd-4f9b-a73e-615958e86cc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.407452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff427ff-b3dd-4f9b-a73e-615958e86cc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.409758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff427ff-b3dd-4f9b-a73e-615958e86cc4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.459781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0a153-7816-4cef-aef3-9f551570ea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.460014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 822/1618] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:36.460137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0a153-7816-4cef-aef3-9f551570ea46 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.460137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0a153-7816-4cef-aef3-9f551570ea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.460137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0a153-7816-4cef-aef3-9f551570ea46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.460137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0a153-7816-4cef-aef3-9f551570ea46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.460555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff427ff-b3dd-4f9b-a73e-615958e86cc4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.462748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0a153-7816-4cef-aef3-9f551570ea46 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.534772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 797/1619] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/projects => generated 387 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:36.534772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0a153-7816-4cef-aef3-9f551570ea46 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.578536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff427ff-b3dd-4f9b-a73e-615958e86cc4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.647449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:36.659021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0a153-7816-4cef-aef3-9f551570ea46 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.685302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418103a7-fe4e-4ec9-a3ba-aa848c3bea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.687552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418103a7-fe4e-4ec9-a3ba-aa848c3bea39 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.689063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418103a7-fe4e-4ec9-a3ba-aa848c3bea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.690359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418103a7-fe4e-4ec9-a3ba-aa848c3bea39 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.691305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418103a7-fe4e-4ec9-a3ba-aa848c3bea39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.697207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418103a7-fe4e-4ec9-a3ba-aa848c3bea39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.705757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:36.716525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ccb9a-1344-4384-b075-1d44e01ea68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.718962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ccb9a-1344-4384-b075-1d44e01ea68c admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.741567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ccb9a-1344-4384-b075-1d44e01ea68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.812961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ccb9a-1344-4384-b075-1d44e01ea68c admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.812961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ccb9a-1344-4384-b075-1d44e01ea68c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.812961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ccb9a-1344-4384-b075-1d44e01ea68c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.814013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 798/1620] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:36.814013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418103a7-fe4e-4ec9-a3ba-aa848c3bea39 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.814013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418103a7-fe4e-4ec9-a3ba-aa848c3bea39 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.814597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 823/1621] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/users => generated 294 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:36.821632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ccb9a-1344-4384-b075-1d44e01ea68c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.827459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccb5a5-72a7-432c-8ac3-8865d8ee5a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.828367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccb5a5-72a7-432c-8ac3-8865d8ee5a63 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.830787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccb5a5-72a7-432c-8ac3-8865d8ee5a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.831488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccb5a5-72a7-432c-8ac3-8865d8ee5a63 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.834762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccb5a5-72a7-432c-8ac3-8865d8ee5a63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.840652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccb5a5-72a7-432c-8ac3-8865d8ee5a63 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.855707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ccb9a-1344-4384-b075-1d44e01ea68c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.867072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f561a28-e3c4-44ed-829a-bdb5eba4ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.869005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f561a28-e3c4-44ed-829a-bdb5eba4ca3d admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.869288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f561a28-e3c4-44ed-829a-bdb5eba4ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.869601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f561a28-e3c4-44ed-829a-bdb5eba4ca3d admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.871205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f561a28-e3c4-44ed-829a-bdb5eba4ca3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.873971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f561a28-e3c4-44ed-829a-bdb5eba4ca3d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.896749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 799/1622] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:36.898206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccb5a5-72a7-432c-8ac3-8865d8ee5a63 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.921382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccb5a5-72a7-432c-8ac3-8865d8ee5a63 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.922967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 824/1623] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:36 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:36.924396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f561a28-e3c4-44ed-829a-bdb5eba4ca3d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.930244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13163-00c9-4a84-81a3-d5f40c60b5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.931755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13163-00c9-4a84-81a3-d5f40c60b5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.932102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13163-00c9-4a84-81a3-d5f40c60b5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.932550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d13163-00c9-4a84-81a3-d5f40c60b5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.933390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d13163-00c9-4a84-81a3-d5f40c60b5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.935570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d13163-00c9-4a84-81a3-d5f40c60b5c2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.956421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f561a28-e3c4-44ed-829a-bdb5eba4ca3d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.964060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 800/1624] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:36.964060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13163-00c9-4a84-81a3-d5f40c60b5c2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:36.972418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553aef0-8593-49e9-89a5-b562520a6791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.973037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553aef0-8593-49e9-89a5-b562520a6791 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.973479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553aef0-8593-49e9-89a5-b562520a6791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:36.973773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553aef0-8593-49e9-89a5-b562520a6791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:36.974888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553aef0-8593-49e9-89a5-b562520a6791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:36.977231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553aef0-8593-49e9-89a5-b562520a6791 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:36.985744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d13163-00c9-4a84-81a3-d5f40c60b5c2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:36.997287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e296345-452b-4042-aca8-798935c7c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:36.998683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e296345-452b-4042-aca8-798935c7c0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:36.999245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e296345-452b-4042-aca8-798935c7c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.000159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 825/1625] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:37.001031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553aef0-8593-49e9-89a5-b562520a6791 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.001295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e296345-452b-4042-aca8-798935c7c0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.002843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e296345-452b-4042-aca8-798935c7c0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.005664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e296345-452b-4042-aca8-798935c7c0a9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.019408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553aef0-8593-49e9-89a5-b562520a6791 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.022005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 801/1626] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:37.023770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e296345-452b-4042-aca8-798935c7c0a9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.033021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791522e-459d-4974-a490-01894144714e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.033704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791522e-459d-4974-a490-01894144714e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.033987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791522e-459d-4974-a490-01894144714e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.034259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791522e-459d-4974-a490-01894144714e admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.035064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791522e-459d-4974-a490-01894144714e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.038940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791522e-459d-4974-a490-01894144714e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.054075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e296345-452b-4042-aca8-798935c7c0a9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.057824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 826/1627] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:37.060766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791522e-459d-4974-a490-01894144714e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.077338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791522e-459d-4974-a490-01894144714e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.081704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd80bb-455f-4482-b2dc-faee73c41f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.082587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd80bb-455f-4482-b2dc-faee73c41f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.083238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd80bb-455f-4482-b2dc-faee73c41f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.084026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd80bb-455f-4482-b2dc-faee73c41f77 admin admin] PATH_INFO: `/v3/projects/061a46be4e7a43a599cea00f1957263c/users/d75daa2d2e20487eaa894a5045277b8c/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.084445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ad67f-80c1-4dc9-9e95-91151f908bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.085257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ad67f-80c1-4dc9-9e95-91151f908bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.085641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ad67f-80c1-4dc9-9e95-91151f908bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.085985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ad67f-80c1-4dc9-9e95-91151f908bf1 admin admin] PATH_INFO: `/v3/projects/ade5fe83294847c78981650d0113d10c/users/d53c8f67066047cf957eefb5e49bdf9b/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.107717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd80bb-455f-4482-b2dc-faee73c41f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061a46be4e7a43a599cea00f1957263c, user_id=d75daa2d2e20487eaa894a5045277b8c, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.109438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ad67f-80c1-4dc9-9e95-91151f908bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade5fe83294847c78981650d0113d10c, user_id=d53c8f67066047cf957eefb5e49bdf9b, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.111396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd80bb-455f-4482-b2dc-faee73c41f77 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.114264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ad67f-80c1-4dc9-9e95-91151f908bf1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.135578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 802/1628] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:37 2022] PUT /identity/v3/projects/061a46be4e7a43a599cea00f1957263c/users/d75daa2d2e20487eaa894a5045277b8c/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:37.136554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd80bb-455f-4482-b2dc-faee73c41f77 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.139328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 827/1629] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:37 2022] PUT /identity/v3/projects/ade5fe83294847c78981650d0113d10c/users/d53c8f67066047cf957eefb5e49bdf9b/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:37.140305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ad67f-80c1-4dc9-9e95-91151f908bf1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.235066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ad67f-80c1-4dc9-9e95-91151f908bf1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.242243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea62cd4-50e1-4842-8f25-5d485e997e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.242978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62cd4-50e1-4842-8f25-5d485e997e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.243405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62cd4-50e1-4842-8f25-5d485e997e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.243677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62cd4-50e1-4842-8f25-5d485e997e9e admin admin] PATH_INFO: `/v3/projects/6f6f6abf4b234c4995c4e2bf6c21568e/users/0e507f69358843a0b50f9d7d54d5bb36/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.264017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea62cd4-50e1-4842-8f25-5d485e997e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6f6abf4b234c4995c4e2bf6c21568e, user_id=0e507f69358843a0b50f9d7d54d5bb36, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.267524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea62cd4-50e1-4842-8f25-5d485e997e9e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.283632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd80bb-455f-4482-b2dc-faee73c41f77 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.294658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 828/1630] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:37 2022] PUT /identity/v3/projects/6f6f6abf4b234c4995c4e2bf6c21568e/users/0e507f69358843a0b50f9d7d54d5bb36/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:37.295741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea62cd4-50e1-4842-8f25-5d485e997e9e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.375856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7b0c-ce7e-47c0-bff3-a7c3ee892471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.378766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7b0c-ce7e-47c0-bff3-a7c3ee892471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.380625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7b0c-ce7e-47c0-bff3-a7c3ee892471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.381133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7b0c-ce7e-47c0-bff3-a7c3ee892471 admin admin] PATH_INFO: `/v3/projects/200c15da9b804a188712f2874ca07393/users/1617f1e5439342ae8177eb2f5f815df5/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.406975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a7b0c-ce7e-47c0-bff3-a7c3ee892471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200c15da9b804a188712f2874ca07393, user_id=1617f1e5439342ae8177eb2f5f815df5, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.411229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a7b0c-ce7e-47c0-bff3-a7c3ee892471 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.447852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea62cd4-50e1-4842-8f25-5d485e997e9e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.451230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 803/1631] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:37 2022] PUT /identity/v3/projects/200c15da9b804a188712f2874ca07393/users/1617f1e5439342ae8177eb2f5f815df5/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:37.451230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7b0c-ce7e-47c0-bff3-a7c3ee892471 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.474425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:37.474758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:37.564003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d67b2-b889-4ac7-bf9d-59ba771458ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.564653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d67b2-b889-4ac7-bf9d-59ba771458ab admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.565015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d67b2-b889-4ac7-bf9d-59ba771458ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.565445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d67b2-b889-4ac7-bf9d-59ba771458ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.571522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d67b2-b889-4ac7-bf9d-59ba771458ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.573141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d67b2-b889-4ac7-bf9d-59ba771458ab admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.581296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7b0c-ce7e-47c0-bff3-a7c3ee892471 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.590311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd8357-7ae9-4d56-924d-b61bafbae008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.591232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8357-7ae9-4d56-924d-b61bafbae008 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.591714 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 829/1632] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:37.593348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d67b2-b889-4ac7-bf9d-59ba771458ab admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.597443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8357-7ae9-4d56-924d-b61bafbae008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.598053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8357-7ae9-4d56-924d-b61bafbae008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.599818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd8357-7ae9-4d56-924d-b61bafbae008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.603825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd8357-7ae9-4d56-924d-b61bafbae008 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.611154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d67b2-b889-4ac7-bf9d-59ba771458ab admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.617495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6466ee-152c-40ed-9e9f-31acbdee21b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.625696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6466ee-152c-40ed-9e9f-31acbdee21b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.626565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6466ee-152c-40ed-9e9f-31acbdee21b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.627025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6466ee-152c-40ed-9e9f-31acbdee21b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.628123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 804/1633] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:37.629036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6466ee-152c-40ed-9e9f-31acbdee21b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.630058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd8357-7ae9-4d56-924d-b61bafbae008 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.631851 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6466ee-152c-40ed-9e9f-31acbdee21b0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.652223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 830/1634] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:37.653487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6466ee-152c-40ed-9e9f-31acbdee21b0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.656411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd8357-7ae9-4d56-924d-b61bafbae008 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.661851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeabdcf-0b9a-43f7-93bc-46b6018f00c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.662579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeabdcf-0b9a-43f7-93bc-46b6018f00c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.662946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeabdcf-0b9a-43f7-93bc-46b6018f00c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.663285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeabdcf-0b9a-43f7-93bc-46b6018f00c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.664039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeabdcf-0b9a-43f7-93bc-46b6018f00c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.665807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeabdcf-0b9a-43f7-93bc-46b6018f00c1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.680634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 805/1635] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:37.681970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeabdcf-0b9a-43f7-93bc-46b6018f00c1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.688202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6466ee-152c-40ed-9e9f-31acbdee21b0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.695814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeabdcf-0b9a-43f7-93bc-46b6018f00c1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.701829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644512a1-d3c7-47e5-844f-2fa0b2b95b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.702677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644512a1-d3c7-47e5-844f-2fa0b2b95b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.703110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644512a1-d3c7-47e5-844f-2fa0b2b95b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.703513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644512a1-d3c7-47e5-844f-2fa0b2b95b20 admin admin] PATH_INFO: `/v3/projects/ade5fe83294847c78981650d0113d10c/users/d53c8f67066047cf957eefb5e49bdf9b/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.705915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644512a1-d3c7-47e5-844f-2fa0b2b95b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade5fe83294847c78981650d0113d10c, user_id=d53c8f67066047cf957eefb5e49bdf9b, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.707552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e752ae8-8af6-4e61-b809-a79b0521cc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.708639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e752ae8-8af6-4e61-b809-a79b0521cc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.709121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e752ae8-8af6-4e61-b809-a79b0521cc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.709933 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e752ae8-8af6-4e61-b809-a79b0521cc88 admin admin] PATH_INFO: `/v3/projects/061a46be4e7a43a599cea00f1957263c/users/d75daa2d2e20487eaa894a5045277b8c/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.712531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e752ae8-8af6-4e61-b809-a79b0521cc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061a46be4e7a43a599cea00f1957263c, user_id=d75daa2d2e20487eaa894a5045277b8c, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.712834 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644512a1-d3c7-47e5-844f-2fa0b2b95b20 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.716304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e752ae8-8af6-4e61-b809-a79b0521cc88 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.733225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 831/1636] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:37 2022] PUT /identity/v3/projects/061a46be4e7a43a599cea00f1957263c/users/d75daa2d2e20487eaa894a5045277b8c/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:37.734464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e752ae8-8af6-4e61-b809-a79b0521cc88 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.739211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 806/1637] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:37 2022] PUT /identity/v3/projects/ade5fe83294847c78981650d0113d10c/users/d53c8f67066047cf957eefb5e49bdf9b/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:37.742032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644512a1-d3c7-47e5-844f-2fa0b2b95b20 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:37.887661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e752ae8-8af6-4e61-b809-a79b0521cc88 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.897680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992da4e-1310-4f56-be0f-1fdf756b56e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.899091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992da4e-1310-4f56-be0f-1fdf756b56e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.899582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992da4e-1310-4f56-be0f-1fdf756b56e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.899979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992da4e-1310-4f56-be0f-1fdf756b56e9 admin admin] PATH_INFO: `/v3/projects/6f6f6abf4b234c4995c4e2bf6c21568e/users/0e507f69358843a0b50f9d7d54d5bb36/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.903207 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992da4e-1310-4f56-be0f-1fdf756b56e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6f6abf4b234c4995c4e2bf6c21568e, user_id=0e507f69358843a0b50f9d7d54d5bb36, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.909090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992da4e-1310-4f56-be0f-1fdf756b56e9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.916205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644512a1-d3c7-47e5-844f-2fa0b2b95b20 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:37.927849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dad90-1d84-4cbb-8961-327367770ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:37.928534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dad90-1d84-4cbb-8961-327367770ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.928814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dad90-1d84-4cbb-8961-327367770ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.929093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dad90-1d84-4cbb-8961-327367770ddc admin admin] PATH_INFO: `/v3/projects/200c15da9b804a188712f2874ca07393/users/1617f1e5439342ae8177eb2f5f815df5/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.935352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 832/1638] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:37 2022] PUT /identity/v3/projects/6f6f6abf4b234c4995c4e2bf6c21568e/users/0e507f69358843a0b50f9d7d54d5bb36/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:37.940334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938f5c3-9ffd-4d1c-90cd-b115f25681c4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.940762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938f5c3-9ffd-4d1c-90cd-b115f25681c4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.941388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938f5c3-9ffd-4d1c-90cd-b115f25681c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:37.942112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dad90-1d84-4cbb-8961-327367770ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=200c15da9b804a188712f2874ca07393, user_id=1617f1e5439342ae8177eb2f5f815df5, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:37.945694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dad90-1d84-4cbb-8961-327367770ddc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:37.970025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 807/1639] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:37 2022] PUT /identity/v3/projects/200c15da9b804a188712f2874ca07393/users/1617f1e5439342ae8177eb2f5f815df5/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:37.971732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a81036-3678-4093-a05c-c6eb46b47bc5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:37.972082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a81036-3678-4093-a05c-c6eb46b47bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:37.972424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a81036-3678-4093-a05c-c6eb46b47bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:38.025611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8938f5c3-9ffd-4d1c-90cd-b115f25681c4 None None] MFA Rules not processed for user `d75daa2d2e20487eaa894a5045277b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:38.030322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8938f5c3-9ffd-4d1c-90cd-b115f25681c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:38.062776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-92a81036-3678-4093-a05c-c6eb46b47bc5 None None] MFA Rules not processed for user `d53c8f67066047cf957eefb5e49bdf9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:38.062776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-92a81036-3678-4093-a05c-c6eb46b47bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:38.196032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 833/1640] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:38.197844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e02ca-f6bf-494a-8a78-2fbf904a9bec None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:38.197844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e02ca-f6bf-494a-8a78-2fbf904a9bec None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:38.198123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e02ca-f6bf-494a-8a78-2fbf904a9bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:38.207209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:38.226237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 808/1641] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:38.228598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d596d-5d38-4361-998c-7eb5122438fa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:38.229839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d596d-5d38-4361-998c-7eb5122438fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:38.230638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d596d-5d38-4361-998c-7eb5122438fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:38.244279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bb3e02ca-f6bf-494a-8a78-2fbf904a9bec None None] MFA Rules not processed for user `0e507f69358843a0b50f9d7d54d5bb36`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:38.245925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bb3e02ca-f6bf-494a-8a78-2fbf904a9bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:38.251851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:38.328915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-418d596d-5d38-4361-998c-7eb5122438fa None None] MFA Rules not processed for user `1617f1e5439342ae8177eb2f5f815df5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:38.333290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-418d596d-5d38-4361-998c-7eb5122438fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:38.377216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 834/1642] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:38.378308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e02ca-f6bf-494a-8a78-2fbf904a9bec None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:38.481362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e02ca-f6bf-494a-8a78-2fbf904a9bec None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:38.486165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954963d7-c35d-435d-b6a6-463e65956a55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:38.486977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954963d7-c35d-435d-b6a6-463e65956a55 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:38.487692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954963d7-c35d-435d-b6a6-463e65956a55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:38.487692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954963d7-c35d-435d-b6a6-463e65956a55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:38.502673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954963d7-c35d-435d-b6a6-463e65956a55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:38.505419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954963d7-c35d-435d-b6a6-463e65956a55 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:38.526387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 835/1643] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:38.527616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954963d7-c35d-435d-b6a6-463e65956a55 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:38.541227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954963d7-c35d-435d-b6a6-463e65956a55 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:38.546460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67f0e8-3790-409d-946d-1da665d45c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:38.547703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67f0e8-3790-409d-946d-1da665d45c30 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:38.547703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67f0e8-3790-409d-946d-1da665d45c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:38.557728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67f0e8-3790-409d-946d-1da665d45c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:38.603418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 809/1644] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:38.655050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67f0e8-3790-409d-946d-1da665d45c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:38.657195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67f0e8-3790-409d-946d-1da665d45c30 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:38.673918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 836/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:38.701755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d596d-5d38-4361-998c-7eb5122438fa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:38.722287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d596d-5d38-4361-998c-7eb5122438fa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:38.735431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87fbbe-035b-4c7d-b7c8-1451075ee5f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:38.742680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87fbbe-035b-4c7d-b7c8-1451075ee5f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:38.742680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87fbbe-035b-4c7d-b7c8-1451075ee5f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:38.742680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87fbbe-035b-4c7d-b7c8-1451075ee5f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:38.776111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87fbbe-035b-4c7d-b7c8-1451075ee5f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:38.778296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87fbbe-035b-4c7d-b7c8-1451075ee5f1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:38.817427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 810/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:38.874187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67f0e8-3790-409d-946d-1da665d45c30 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:38.882574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87fbbe-035b-4c7d-b7c8-1451075ee5f1 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:38.893574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87fbbe-035b-4c7d-b7c8-1451075ee5f1 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:38.899493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ab56a-5f18-4128-9211-36f9edc49cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:38.900374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ab56a-5f18-4128-9211-36f9edc49cca service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:38.901525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ab56a-5f18-4128-9211-36f9edc49cca service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:38.902548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ab56a-5f18-4128-9211-36f9edc49cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:38.915943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ab56a-5f18-4128-9211-36f9edc49cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:38.920898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ab56a-5f18-4128-9211-36f9edc49cca service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:38.958604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 811/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:38.972408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:39.090339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67f0e8-3790-409d-946d-1da665d45c30 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:39.104504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:39.107016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:39.107431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:39.108885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] PATH_INFO: `/v3/users/05d7ba531c534b76a1b214d6501f49b2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:39.113029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d7ba531c534b76a1b214d6501f49b2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:39.116231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:34:39.116231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Aug 31 04:34:39.116231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: Aug 31 04:34:39.118346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:39.290163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ab56a-5f18-4128-9211-36f9edc49cca service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:39.299736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ab56a-5f18-4128-9211-36f9edc49cca service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:39.301420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d7ba531c534b76a1b214d6501f49b2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:39.316242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f601a11-620d-430b-934b-9684263e12ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:39.316873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f601a11-620d-430b-934b-9684263e12ca service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:39.317192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f601a11-620d-430b-934b-9684263e12ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:39.317506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f601a11-620d-430b-934b-9684263e12ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:39.325614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d7ba531c534b76a1b214d6501f49b2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:39.326335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d7ba531c534b76a1b214d6501f49b2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:39.377125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d7ba531c534b76a1b214d6501f49b2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:39.377995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-569fe88d-b8f7-4d02-b3e5-56ac89603a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d7ba531c534b76a1b214d6501f49b2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:39.407314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 837/1648] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:34:38 2022] DELETE /identity/v3/users/05d7ba531c534b76a1b214d6501f49b2 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:39.410242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51bd1-524a-4778-844a-568868807b34 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:39.412769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51bd1-524a-4778-844a-568868807b34 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:39.413149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51bd1-524a-4778-844a-568868807b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:39.439833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-deb51bd1-524a-4778-844a-568868807b34 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:39.441256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-deb51bd1-524a-4778-844a-568868807b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:39.484706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f601a11-620d-430b-934b-9684263e12ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:39.486189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f601a11-620d-430b-934b-9684263e12ca service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:39.513221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 812/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:34:39.674895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 838/1650] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:39.785506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f601a11-620d-430b-934b-9684263e12ca service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:39.896020 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f601a11-620d-430b-934b-9684263e12ca service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:39.920668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:39.996488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176785ff-f892-4d92-8b6b-d7daca1384e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:39.997123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176785ff-f892-4d92-8b6b-d7daca1384e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:39.997399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176785ff-f892-4d92-8b6b-d7daca1384e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:39.997665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176785ff-f892-4d92-8b6b-d7daca1384e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:40.015193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176785ff-f892-4d92-8b6b-d7daca1384e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:40.016557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176785ff-f892-4d92-8b6b-d7daca1384e1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:40.034638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 813/1651] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:40.319928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51bd1-524a-4778-844a-568868807b34 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:40.321536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-deb51bd1-524a-4778-844a-568868807b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:40.334015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51bd1-524a-4778-844a-568868807b34 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:40.338978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:40.339535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:40.339881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:40.340160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] PATH_INFO: `/v3/users/23dd1dbd89174c8092cb59e9b021967c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:40.341329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=23dd1dbd89174c8092cb59e9b021967c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:40.344368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:40.454249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:40.456499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23dd1dbd89174c8092cb59e9b021967c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:40.472307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23dd1dbd89174c8092cb59e9b021967c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:40.473099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23dd1dbd89174c8092cb59e9b021967c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:40.491790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23dd1dbd89174c8092cb59e9b021967c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:40.492644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23dd1dbd89174c8092cb59e9b021967c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:40.511382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 839/1652] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:34:40 2022] DELETE /identity/v3/users/23dd1dbd89174c8092cb59e9b021967c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:40.513216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddb016-07d6-4cdd-b1d4-8bbad1e4c55e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:40.513605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddb016-07d6-4cdd-b1d4-8bbad1e4c55e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:40.513884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddb016-07d6-4cdd-b1d4-8bbad1e4c55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:40.553589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9cddb016-07d6-4cdd-b1d4-8bbad1e4c55e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:40.555004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9cddb016-07d6-4cdd-b1d4-8bbad1e4c55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:40.675372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 814/1653] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:34:40.721035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:40.723987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:40.894054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091afeca-0e2a-40d3-beb2-2c621c427c3b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:40.910369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757cb3f-db48-49e4-bb9d-6357f45eff51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:40.911141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757cb3f-db48-49e4-bb9d-6357f45eff51 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:40.911451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757cb3f-db48-49e4-bb9d-6357f45eff51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:40.911716 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757cb3f-db48-49e4-bb9d-6357f45eff51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:40.930504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757cb3f-db48-49e4-bb9d-6357f45eff51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:40.932054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757cb3f-db48-49e4-bb9d-6357f45eff51 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:40.956090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 840/1654] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:42.123564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddb016-07d6-4cdd-b1d4-8bbad1e4c55e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:42.125019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9cddb016-07d6-4cdd-b1d4-8bbad1e4c55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:42.140955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddb016-07d6-4cdd-b1d4-8bbad1e4c55e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:42.153896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:42.154570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:42.154931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:42.155206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] PATH_INFO: `/v3/projects/3fd095529d224079ac9f991ca414526b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:42.156952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd095529d224079ac9f991ca414526b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:42.157994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:34:42.157994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 31 04:34:42.157994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: Aug 31 04:34:42.159278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:42.269665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd095529d224079ac9f991ca414526b', 'request_id': 'req-ba45d679-d6d4-473a-8772-0eeb6d49727a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.291671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd095529d224079ac9f991ca414526b', 'request_id': 'req-ba45d679-d6d4-473a-8772-0eeb6d49727a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.308290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd095529d224079ac9f991ca414526b', 'request_id': 'req-ba45d679-d6d4-473a-8772-0eeb6d49727a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.330023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] The token cache is being invalidate because project 3fd095529d224079ac9f991ca414526b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:34:42.330642 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.331922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.335220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 815/1655] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:34:42 2022] DELETE /identity/v3/projects/3fd095529d224079ac9f991ca414526b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:42.340685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757cb3f-db48-49e4-bb9d-6357f45eff51 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:42.353065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757cb3f-db48-49e4-bb9d-6357f45eff51 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:42.543705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:42.548487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:42.548981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:42.549465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] PATH_INFO: `/v3/users/d75daa2d2e20487eaa894a5045277b8c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:42.562806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75daa2d2e20487eaa894a5045277b8c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:42.570648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:42.722727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75daa2d2e20487eaa894a5045277b8c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.749766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75daa2d2e20487eaa894a5045277b8c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.750746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75daa2d2e20487eaa894a5045277b8c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.806252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:42.811441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75daa2d2e20487eaa894a5045277b8c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.819333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75daa2d2e20487eaa894a5045277b8c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:42.847434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 841/1656] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:34:42 2022] DELETE /identity/v3/users/d75daa2d2e20487eaa894a5045277b8c => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:43.033028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:43.034577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:43.056928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45d679-d6d4-473a-8772-0eeb6d49727a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:43.173851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:43.204014 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:43.205377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:43.205800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:43.207034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] PATH_INFO: `/v3/projects/c6907a65652e4be9ab958617329e7484` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:43.209703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6907a65652e4be9ab958617329e7484)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:43.212264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:43.331848 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6907a65652e4be9ab958617329e7484', 'request_id': 'req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.353979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6907a65652e4be9ab958617329e7484', 'request_id': 'req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.373021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6907a65652e4be9ab958617329e7484', 'request_id': 'req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.397909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] The token cache is being invalidate because project c6907a65652e4be9ab958617329e7484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:34:43.398265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.400089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ebf66d0-a0e1-4c1f-9e77-1b086c47ccd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.406422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 816/1657] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:34:43 2022] DELETE /identity/v3/projects/c6907a65652e4be9ab958617329e7484 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:43.410024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:43.412522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:43.450292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed8a95-f112-4bb2-a1d7-631413557928 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:43.566475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:43.567192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:43.567542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:43.568517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] PATH_INFO: `/v3/users/0e507f69358843a0b50f9d7d54d5bb36` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:43.573087 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e507f69358843a0b50f9d7d54d5bb36)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:43.575397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:43.675572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e507f69358843a0b50f9d7d54d5bb36'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.690802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e507f69358843a0b50f9d7d54d5bb36'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.691516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e507f69358843a0b50f9d7d54d5bb36'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.728071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e507f69358843a0b50f9d7d54d5bb36'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.728799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e507f69358843a0b50f9d7d54d5bb36'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:43.756607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 842/1658] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:34:43 2022] DELETE /identity/v3/users/0e507f69358843a0b50f9d7d54d5bb36 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:44.643739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201963ca-83d2-4b69-8393-1963e9889f60 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:44.645558 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201963ca-83d2-4b69-8393-1963e9889f60 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:44.645871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201963ca-83d2-4b69-8393-1963e9889f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:44.675258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:44.677493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:44.715229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-201963ca-83d2-4b69-8393-1963e9889f60 None None] MFA Rules not processed for user `0f863757bba840ea85db119179b9fc7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:44.717686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-201963ca-83d2-4b69-8393-1963e9889f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:44.842710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e247b8-9f2f-47c4-93c3-b280e7da3e63 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:44.848935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:44.850113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:44.850619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:44.851581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] PATH_INFO: `/v3/projects/061a46be4e7a43a599cea00f1957263c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:44.853954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061a46be4e7a43a599cea00f1957263c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:44.857908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:44.867495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 817/1659] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:44.869108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc101c-0524-4681-b70f-d29af453c84f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:44.869507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc101c-0524-4681-b70f-d29af453c84f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:44.869849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc101c-0524-4681-b70f-d29af453c84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:44.898704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4fdc101c-0524-4681-b70f-d29af453c84f None None] MFA Rules not processed for user `0f863757bba840ea85db119179b9fc7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:44.900236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4fdc101c-0524-4681-b70f-d29af453c84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:44.901410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:44.921705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 818/1660] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:44.951203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddb11f-f09d-4a66-baa1-55ffd2e16929 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:44.951623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddb11f-f09d-4a66-baa1-55ffd2e16929 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:44.951915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddb11f-f09d-4a66-baa1-55ffd2e16929 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:44.954263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 819/1661] 10.222.0.42 () {54 vars in 962 bytes} [Wed Aug 31 04:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:34:44.963224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fcd5c-7f31-42a5-af15-e95c5b14cc61 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:44.963594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fcd5c-7f31-42a5-af15-e95c5b14cc61 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:44.963968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fcd5c-7f31-42a5-af15-e95c5b14cc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:44.995778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061a46be4e7a43a599cea00f1957263c', 'request_id': 'req-bac5b69d-715d-4498-b28e-0150fa621995'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.015552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061a46be4e7a43a599cea00f1957263c', 'request_id': 'req-bac5b69d-715d-4498-b28e-0150fa621995'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.043530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061a46be4e7a43a599cea00f1957263c', 'request_id': 'req-bac5b69d-715d-4498-b28e-0150fa621995'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.054873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fb9fcd5c-7f31-42a5-af15-e95c5b14cc61 None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:45.056294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb9fcd5c-7f31-42a5-af15-e95c5b14cc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:45.067161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] The token cache is being invalidate because project 061a46be4e7a43a599cea00f1957263c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:34:45.067866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.070325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bac5b69d-715d-4498-b28e-0150fa621995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.074641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 843/1662] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:34:44 2022] DELETE /identity/v3/projects/061a46be4e7a43a599cea00f1957263c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:34:45.093797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b93d5-4fab-46cd-b11e-77fce4fdc4b2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.094381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b93d5-4fab-46cd-b11e-77fce4fdc4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.094845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b93d5-4fab-46cd-b11e-77fce4fdc4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.161240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:45.173525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-af3b93d5-4fab-46cd-b11e-77fce4fdc4b2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:45.175078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-af3b93d5-4fab-46cd-b11e-77fce4fdc4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:45.236502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 820/1663] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Aug 31 04:34:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:45.237405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fcd5c-7f31-42a5-af15-e95c5b14cc61 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:45.238847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb9fcd5c-7f31-42a5-af15-e95c5b14cc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:45.335031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 844/1664] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:45.336975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b93d5-4fab-46cd-b11e-77fce4fdc4b2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:45.360128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b93d5-4fab-46cd-b11e-77fce4fdc4b2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:45.366614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ece777-962a-47f9-8c34-5c2bbf84ee55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:45.367932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece777-962a-47f9-8c34-5c2bbf84ee55 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.368303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece777-962a-47f9-8c34-5c2bbf84ee55 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.368674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece777-962a-47f9-8c34-5c2bbf84ee55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.371713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e3ece777-962a-47f9-8c34-5c2bbf84ee55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:45.383443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fcd5c-7f31-42a5-af15-e95c5b14cc61 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:45.390881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:45.391550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.391831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.392265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] PATH_INFO: `/v3/projects/6f6f6abf4b234c4995c4e2bf6c21568e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.393702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6f6abf4b234c4995c4e2bf6c21568e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:45.396116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:45.482666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6f6abf4b234c4995c4e2bf6c21568e', 'request_id': 'req-42403268-59c9-443b-a029-7e77d8639c12'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.501918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6f6abf4b234c4995c4e2bf6c21568e', 'request_id': 'req-42403268-59c9-443b-a029-7e77d8639c12'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.508538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ece777-962a-47f9-8c34-5c2bbf84ee55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:45.511206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ece777-962a-47f9-8c34-5c2bbf84ee55 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:45.522200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6f6abf4b234c4995c4e2bf6c21568e', 'request_id': 'req-42403268-59c9-443b-a029-7e77d8639c12'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.533472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 845/1665] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:45.538835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] The token cache is being invalidate because project 6f6f6abf4b234c4995c4e2bf6c21568e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:34:45.539277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.539998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42403268-59c9-443b-a029-7e77d8639c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:45.545594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 821/1666] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:34:45 2022] DELETE /identity/v3/projects/6f6f6abf4b234c4995c4e2bf6c21568e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:45.547123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a32ef2-e3c0-4227-bd0b-e53258085d65 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.547612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a32ef2-e3c0-4227-bd0b-e53258085d65 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.547892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a32ef2-e3c0-4227-bd0b-e53258085d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.562626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d8a96-d11f-4fd9-b5c8-805296a45810 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.563046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d8a96-d11f-4fd9-b5c8-805296a45810 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.563565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d8a96-d11f-4fd9-b5c8-805296a45810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.577166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-25a32ef2-e3c0-4227-bd0b-e53258085d65 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:45.578698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-25a32ef2-e3c0-4227-bd0b-e53258085d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:45.625034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a63d8a96-d11f-4fd9-b5c8-805296a45810 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:45.629453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a63d8a96-d11f-4fd9-b5c8-805296a45810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:45.695290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 822/1667] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:45.704467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b65c54-c701-462f-99c0-0e125cf449dc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.706789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b65c54-c701-462f-99c0-0e125cf449dc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.709377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b65c54-c701-462f-99c0-0e125cf449dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.727860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:45.776833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:45.795672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-34b65c54-c701-462f-99c0-0e125cf449dc None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:45.803233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34b65c54-c701-462f-99c0-0e125cf449dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:45.829158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 823/1668] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:45.830095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b65c54-c701-462f-99c0-0e125cf449dc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:45.840208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 846/1669] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:45.840557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b65c54-c701-462f-99c0-0e125cf449dc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:45.841238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d8a96-d11f-4fd9-b5c8-805296a45810 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:45.846080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:45.846729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.847025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.847295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.847967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:45.848272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:45.852024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:45.865318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 824/1670] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:45.870124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:45.871474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d8a96-d11f-4fd9-b5c8-805296a45810 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:45.883504 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e3196-30ac-43f2-81e4-f49e765b8cb8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:45.889609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf40aa4-d563-4651-b961-d0c7003dc350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:45.890900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf40aa4-d563-4651-b961-d0c7003dc350 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.891185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf40aa4-d563-4651-b961-d0c7003dc350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.891285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:45.891457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf40aa4-d563-4651-b961-d0c7003dc350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.892100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf40aa4-d563-4651-b961-d0c7003dc350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:45.894039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:45.894338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:45.894686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf40aa4-d563-4651-b961-d0c7003dc350 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:45.894797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:45.900426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:45.900940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:45.922951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:45.941953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 825/1671] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:45 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:45.943981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf40aa4-d563-4651-b961-d0c7003dc350 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:45.953113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 847/1672] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:45.959339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:45.989388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081015d-fe39-419d-995b-98b06ee893c1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.164908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf40aa4-d563-4651-b961-d0c7003dc350 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.183685 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9ef21-e3f0-433f-a5ff-bc573a562ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.184333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9ef21-e3f0-433f-a5ff-bc573a562ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.184613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9ef21-e3f0-433f-a5ff-bc573a562ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.184887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9ef21-e3f0-433f-a5ff-bc573a562ccb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.185531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9ef21-e3f0-433f-a5ff-bc573a562ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.187525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9ef21-e3f0-433f-a5ff-bc573a562ccb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.194415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b55f0-8c5d-4ce8-9a29-8a60406c3f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.197320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b55f0-8c5d-4ce8-9a29-8a60406c3f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.197612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b55f0-8c5d-4ce8-9a29-8a60406c3f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.197901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b55f0-8c5d-4ce8-9a29-8a60406c3f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.198977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b55f0-8c5d-4ce8-9a29-8a60406c3f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.201683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b55f0-8c5d-4ce8-9a29-8a60406c3f59 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.264937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 826/1673] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:45 2022] POST /identity/v3/users => generated 291 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:46.268921 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9ef21-e3f0-433f-a5ff-bc573a562ccb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:46.278623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 848/1674] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:45 2022] POST /identity/v3/projects => generated 367 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:34:46.282552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b55f0-8c5d-4ce8-9a29-8a60406c3f59 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:46.291456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9ef21-e3f0-433f-a5ff-bc573a562ccb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.404738 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0bc47-a028-422c-b360-2bfc758aaeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.406337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0bc47-a028-422c-b360-2bfc758aaeae admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.406851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0bc47-a028-422c-b360-2bfc758aaeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.407317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0bc47-a028-422c-b360-2bfc758aaeae admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.408323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0bc47-a028-422c-b360-2bfc758aaeae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.410157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0bc47-a028-422c-b360-2bfc758aaeae admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.417355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b55f0-8c5d-4ce8-9a29-8a60406c3f59 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.426600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394cdc1-97ab-4a86-a2c7-d71fd6784a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.427216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394cdc1-97ab-4a86-a2c7-d71fd6784a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.427720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394cdc1-97ab-4a86-a2c7-d71fd6784a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.428906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394cdc1-97ab-4a86-a2c7-d71fd6784a29 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.429720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394cdc1-97ab-4a86-a2c7-d71fd6784a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.430167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 827/1675] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:46.431842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bee2-ba3c-4cfb-ba06-750b9b6a1c06 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.432324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bee2-ba3c-4cfb-ba06-750b9b6a1c06 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.432821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371bee2-ba3c-4cfb-ba06-750b9b6a1c06 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.436812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394cdc1-97ab-4a86-a2c7-d71fd6784a29 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.438846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 828/1676] 10.222.0.42 () {54 vars in 912 bytes} [Wed Aug 31 04:34:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:34:46.440099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bee2-ba3c-4cfb-ba06-750b9b6a1c06 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:46.453951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371bee2-ba3c-4cfb-ba06-750b9b6a1c06 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.459628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d11014-0828-428b-a499-4fb866b11a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.460611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d11014-0828-428b-a499-4fb866b11a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.461084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d11014-0828-428b-a499-4fb866b11a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.463615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d11014-0828-428b-a499-4fb866b11a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.464544 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d11014-0828-428b-a499-4fb866b11a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.466388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d11014-0828-428b-a499-4fb866b11a1a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.479108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 829/1677] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:46.480935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78ec86-ae80-46cb-871e-c5cfce4a1a24 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.481409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78ec86-ae80-46cb-871e-c5cfce4a1a24 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.481877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78ec86-ae80-46cb-871e-c5cfce4a1a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.485750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 849/1678] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:46 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:46.486636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394cdc1-97ab-4a86-a2c7-d71fd6784a29 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:46.504278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394cdc1-97ab-4a86-a2c7-d71fd6784a29 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.508718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7e78ec86-ae80-46cb-871e-c5cfce4a1a24 None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:46.510369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7e78ec86-ae80-46cb-871e-c5cfce4a1a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:46.516227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7cb4c-4891-4097-a63c-8547a45e942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.516934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7cb4c-4891-4097-a63c-8547a45e942b admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.517216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7cb4c-4891-4097-a63c-8547a45e942b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.518390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7cb4c-4891-4097-a63c-8547a45e942b admin admin] PATH_INFO: `/v3/projects/9f1ea53eaafb4650867378af0b21bc4f/users/a4ea075c1b5e4c04bec5643dd44edc9a/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.555521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7cb4c-4891-4097-a63c-8547a45e942b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1ea53eaafb4650867378af0b21bc4f, user_id=a4ea075c1b5e4c04bec5643dd44edc9a, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.558230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7cb4c-4891-4097-a63c-8547a45e942b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.596833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 850/1679] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:46 2022] PUT /identity/v3/projects/9f1ea53eaafb4650867378af0b21bc4f/users/a4ea075c1b5e4c04bec5643dd44edc9a/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:46.598332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7cb4c-4891-4097-a63c-8547a45e942b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:46.599286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:46.628268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:46.702027 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 830/1680] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:34:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:46.705561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e3b3-807a-4e97-8368-50701d702a9b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.706060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e3b3-807a-4e97-8368-50701d702a9b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.706520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e3b3-807a-4e97-8368-50701d702a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.742031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9e78e3b3-807a-4e97-8368-50701d702a9b None None] MFA Rules not processed for user `a4ea075c1b5e4c04bec5643dd44edc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:46.743882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9e78e3b3-807a-4e97-8368-50701d702a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:46.748549 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7cb4c-4891-4097-a63c-8547a45e942b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.765345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.765999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.766280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.766555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.767254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.768940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.787996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 851/1681] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:46.789259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:46.791373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:46.851318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 831/1682] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:46.853804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e3b3-807a-4e97-8368-50701d702a9b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:46.872838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e3b3-807a-4e97-8368-50701d702a9b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.878686 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31a9d-aaf2-453c-bb45-691db10eab87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.879904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31a9d-aaf2-453c-bb45-691db10eab87 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.880517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31a9d-aaf2-453c-bb45-691db10eab87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.881137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31a9d-aaf2-453c-bb45-691db10eab87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.881995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d31a9d-aaf2-453c-bb45-691db10eab87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.884000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d31a9d-aaf2-453c-bb45-691db10eab87 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.899574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 832/1683] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:46.900797 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31a9d-aaf2-453c-bb45-691db10eab87 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:46.923331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31a9d-aaf2-453c-bb45-691db10eab87 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.939406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145757a5-bb29-49fb-8eb7-58e43b8a48ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.942817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145757a5-bb29-49fb-8eb7-58e43b8a48ba admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.943410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145757a5-bb29-49fb-8eb7-58e43b8a48ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.944086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145757a5-bb29-49fb-8eb7-58e43b8a48ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:46.945661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145757a5-bb29-49fb-8eb7-58e43b8a48ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:46.952450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145757a5-bb29-49fb-8eb7-58e43b8a48ba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:46.976872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c29f7d-f331-40f5-bf31-b12f99cc6b59 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:46.986966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971a6c7-99f2-4689-a77b-f339c76bea2d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:46.988462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971a6c7-99f2-4689-a77b-f339c76bea2d service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:46.988963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971a6c7-99f2-4689-a77b-f339c76bea2d service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:46.989878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971a6c7-99f2-4689-a77b-f339c76bea2d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:47.011346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 833/1684] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:46 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:47.015690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145757a5-bb29-49fb-8eb7-58e43b8a48ba admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:47.184296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145757a5-bb29-49fb-8eb7-58e43b8a48ba admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:47.187588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971a6c7-99f2-4689-a77b-f339c76bea2d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:47.189325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971a6c7-99f2-4689-a77b-f339c76bea2d service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:47.199025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:47.199723 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:47.200001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:47.200275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] PATH_INFO: `/v3/projects/51854995213c448882c666ef1cbe2579/users/13ff7867572c48889bba01106b771765/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:47.218163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 852/1685] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:47.222179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971a6c7-99f2-4689-a77b-f339c76bea2d service glance] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:47.234316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] RBAC: Authorizing `identity:create_grant(project_id=51854995213c448882c666ef1cbe2579, user_id=13ff7867572c48889bba01106b771765, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:47.237055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:47.246797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971a6c7-99f2-4689-a77b-f339c76bea2d service glance] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:47.254211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb96e38-149b-4a5f-9a33-7b1d8efc6f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:47.255083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb96e38-149b-4a5f-9a33-7b1d8efc6f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:47.255493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb96e38-149b-4a5f-9a33-7b1d8efc6f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:47.255804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb96e38-149b-4a5f-9a33-7b1d8efc6f1a admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:47.256452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb96e38-149b-4a5f-9a33-7b1d8efc6f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:47.258421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb96e38-149b-4a5f-9a33-7b1d8efc6f1a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:47.277106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 834/1686] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:47 2022] PUT /identity/v3/projects/51854995213c448882c666ef1cbe2579/users/13ff7867572c48889bba01106b771765/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:47.277106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:47.277106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:47.328593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:47.356856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 853/1687] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:47 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:47.361143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ca582-c330-4fea-bc0d-034ef0a9a39d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:47.361437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ca582-c330-4fea-bc0d-034ef0a9a39d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:47.361715 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ca582-c330-4fea-bc0d-034ef0a9a39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:47.413803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:47.427928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-432ca582-c330-4fea-bc0d-034ef0a9a39d None None] MFA Rules not processed for user `13ff7867572c48889bba01106b771765`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:47.429173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-432ca582-c330-4fea-bc0d-034ef0a9a39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:47.440321 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bad4c3-cc2d-4f10-85af-d9a56c56921a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:47.455470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:47.456124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:47.456407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:47.456681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:47.459795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:47.567868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 854/1688] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:47.568842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ca582-c330-4fea-bc0d-034ef0a9a39d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:47.592105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:47.594110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:47.617146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 835/1689] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:47.619415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:47.724615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ca582-c330-4fea-bc0d-034ef0a9a39d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:47.738313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adce39-c050-4989-b0f2-a4680a0fc600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:47.739051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adce39-c050-4989-b0f2-a4680a0fc600 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:47.739371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adce39-c050-4989-b0f2-a4680a0fc600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:47.739643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adce39-c050-4989-b0f2-a4680a0fc600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:47.740340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adce39-c050-4989-b0f2-a4680a0fc600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:47.745679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adce39-c050-4989-b0f2-a4680a0fc600 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:47.771888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25a72-405e-4baf-8039-adda6982e82a service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:47.773622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 855/1690] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:47.779846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adce39-c050-4989-b0f2-a4680a0fc600 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:47.791129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdded9a8-53ab-4198-8557-947aa9d2c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:47.791635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdded9a8-53ab-4198-8557-947aa9d2c70d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:47.792074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdded9a8-53ab-4198-8557-947aa9d2c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:47.792516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdded9a8-53ab-4198-8557-947aa9d2c70d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:47.793276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdded9a8-53ab-4198-8557-947aa9d2c70d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:47.796723 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdded9a8-53ab-4198-8557-947aa9d2c70d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:47.798937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adce39-c050-4989-b0f2-a4680a0fc600 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:47.816349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f7e2f-cf85-4b7f-93a5-caa389acd19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:47.817047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f7e2f-cf85-4b7f-93a5-caa389acd19a admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:47.817325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f7e2f-cf85-4b7f-93a5-caa389acd19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:47.817682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f7e2f-cf85-4b7f-93a5-caa389acd19a admin admin] PATH_INFO: `/v3/projects/82c59067fd754d1aa176231ccd7fcc88/users/4afbe231a9184f0788ecead87afd1d68/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:47.857303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f7e2f-cf85-4b7f-93a5-caa389acd19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c59067fd754d1aa176231ccd7fcc88, user_id=4afbe231a9184f0788ecead87afd1d68, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:47.865895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f7e2f-cf85-4b7f-93a5-caa389acd19a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:47.867080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 836/1691] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:47 2022] POST /identity/v3/projects => generated 369 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:47.870882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdded9a8-53ab-4198-8557-947aa9d2c70d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:47.896902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 856/1692] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:47 2022] PUT /identity/v3/projects/82c59067fd754d1aa176231ccd7fcc88/users/4afbe231a9184f0788ecead87afd1d68/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:47.901494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f7e2f-cf85-4b7f-93a5-caa389acd19a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:47.977276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:48.011274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdded9a8-53ab-4198-8557-947aa9d2c70d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:48.019963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f7e2f-cf85-4b7f-93a5-caa389acd19a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:48.024926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f221eb-ccfa-41c0-8133-96d69db56beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:48.025775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f221eb-ccfa-41c0-8133-96d69db56beb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.028357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f221eb-ccfa-41c0-8133-96d69db56beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.028711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f221eb-ccfa-41c0-8133-96d69db56beb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.029520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f221eb-ccfa-41c0-8133-96d69db56beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:48.034586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50227ec9-5675-4e4e-bb5b-c99d9fb590d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:48.035263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50227ec9-5675-4e4e-bb5b-c99d9fb590d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.035366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f221eb-ccfa-41c0-8133-96d69db56beb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:48.035543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50227ec9-5675-4e4e-bb5b-c99d9fb590d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.035818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50227ec9-5675-4e4e-bb5b-c99d9fb590d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.036791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50227ec9-5675-4e4e-bb5b-c99d9fb590d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:48.038372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50227ec9-5675-4e4e-bb5b-c99d9fb590d4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:48.071369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 857/1693] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:48.072363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50227ec9-5675-4e4e-bb5b-c99d9fb590d4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:48.098919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50227ec9-5675-4e4e-bb5b-c99d9fb590d4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:48.099844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 837/1694] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:34:47 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:48.102531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f221eb-ccfa-41c0-8133-96d69db56beb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:48.109603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d40e6f-f8b1-4f78-b996-29cb765fd6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:48.110310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d40e6f-f8b1-4f78-b996-29cb765fd6e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.110593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d40e6f-f8b1-4f78-b996-29cb765fd6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.113283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d40e6f-f8b1-4f78-b996-29cb765fd6e1 admin admin] PATH_INFO: `/v3/projects/82c59067fd754d1aa176231ccd7fcc88/users/4afbe231a9184f0788ecead87afd1d68/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.118289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d40e6f-f8b1-4f78-b996-29cb765fd6e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c59067fd754d1aa176231ccd7fcc88, user_id=4afbe231a9184f0788ecead87afd1d68, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:48.125854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d40e6f-f8b1-4f78-b996-29cb765fd6e1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:48.129774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f221eb-ccfa-41c0-8133-96d69db56beb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:48.132895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:48.147545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a756e5-91cc-499d-a09b-1a467816ba45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:48.148209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a756e5-91cc-499d-a09b-1a467816ba45 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.148488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a756e5-91cc-499d-a09b-1a467816ba45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.148757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a756e5-91cc-499d-a09b-1a467816ba45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.149480 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a756e5-91cc-499d-a09b-1a467816ba45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:48.151528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a756e5-91cc-499d-a09b-1a467816ba45 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:48.166666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 858/1695] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:48 2022] PUT /identity/v3/projects/82c59067fd754d1aa176231ccd7fcc88/users/4afbe231a9184f0788ecead87afd1d68/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:48.168281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157509e-b83d-4c6d-9062-cc558e45b409 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.168581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157509e-b83d-4c6d-9062-cc558e45b409 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.168853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157509e-b83d-4c6d-9062-cc558e45b409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.181080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 838/1696] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:34:48.183431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a756e5-91cc-499d-a09b-1a467816ba45 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:48.209589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a756e5-91cc-499d-a09b-1a467816ba45 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:48.248588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5157509e-b83d-4c6d-9062-cc558e45b409 None None] MFA Rules not processed for user `4afbe231a9184f0788ecead87afd1d68`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:48.251691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5157509e-b83d-4c6d-9062-cc558e45b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:48.359464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d069cd0-0b5f-4ccc-8cb5-3f78074ed9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:48.360401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d069cd0-0b5f-4ccc-8cb5-3f78074ed9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.360681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d069cd0-0b5f-4ccc-8cb5-3f78074ed9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.360955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d069cd0-0b5f-4ccc-8cb5-3f78074ed9fd admin admin] PATH_INFO: `/v3/projects/e7828485ce8545c7985c966188fd1349/users/91ea94e38e384e9ea072d444c8e4bb26/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.408010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d069cd0-0b5f-4ccc-8cb5-3f78074ed9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7828485ce8545c7985c966188fd1349, user_id=91ea94e38e384e9ea072d444c8e4bb26, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:48.410949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d069cd0-0b5f-4ccc-8cb5-3f78074ed9fd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:48.456195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 839/1697] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:48 2022] PUT /identity/v3/projects/e7828485ce8545c7985c966188fd1349/users/91ea94e38e384e9ea072d444c8e4bb26/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:48.457344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d069cd0-0b5f-4ccc-8cb5-3f78074ed9fd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:48.491691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 859/1698] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:48.509460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157509e-b83d-4c6d-9062-cc558e45b409 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:48.511215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5157509e-b83d-4c6d-9062-cc558e45b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:48.648629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d069cd0-0b5f-4ccc-8cb5-3f78074ed9fd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:48.681251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b914a-e4a9-4bf5-9b7a-4a7f5d8ca4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:48.681890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b914a-e4a9-4bf5-9b7a-4a7f5d8ca4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.682170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b914a-e4a9-4bf5-9b7a-4a7f5d8ca4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.682438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b914a-e4a9-4bf5-9b7a-4a7f5d8ca4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.683175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b914a-e4a9-4bf5-9b7a-4a7f5d8ca4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:48.683457 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157509e-b83d-4c6d-9062-cc558e45b409 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:48.685496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b914a-e4a9-4bf5-9b7a-4a7f5d8ca4e7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:48.719424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 840/1699] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:34:48.723008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e1c05-0fdc-4287-8562-1fb9c631d0ac None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.724276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e1c05-0fdc-4287-8562-1fb9c631d0ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.726037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e1c05-0fdc-4287-8562-1fb9c631d0ac None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.733904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 841/1700] 10.222.0.42 () {54 vars in 912 bytes} [Wed Aug 31 04:34:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:34:48.735601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e1c05-0fdc-4287-8562-1fb9c631d0ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:48.771244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e1c05-0fdc-4287-8562-1fb9c631d0ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:48.782185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d691b8-e195-49b8-ba0c-79f5ea88b765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:48.784216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d691b8-e195-49b8-ba0c-79f5ea88b765 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.784527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d691b8-e195-49b8-ba0c-79f5ea88b765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.784804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d691b8-e195-49b8-ba0c-79f5ea88b765 admin admin] PATH_INFO: `/v3/projects/e7828485ce8545c7985c966188fd1349/users/91ea94e38e384e9ea072d444c8e4bb26/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.797039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d691b8-e195-49b8-ba0c-79f5ea88b765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7828485ce8545c7985c966188fd1349, user_id=91ea94e38e384e9ea072d444c8e4bb26, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:48.797298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d691b8-e195-49b8-ba0c-79f5ea88b765 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:48.832899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 842/1701] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:34:48 2022] PUT /identity/v3/projects/e7828485ce8545c7985c966188fd1349/users/91ea94e38e384e9ea072d444c8e4bb26/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:48.832899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb30c5-9126-411f-a7d1-be575c9fa466 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.832899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb30c5-9126-411f-a7d1-be575c9fa466 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.832899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb30c5-9126-411f-a7d1-be575c9fa466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.856913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b04f62-9efa-4de0-b7c0-8cbba85782b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:48.858350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b04f62-9efa-4de0-b7c0-8cbba85782b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:48.859090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b04f62-9efa-4de0-b7c0-8cbba85782b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:48.859675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b04f62-9efa-4de0-b7c0-8cbba85782b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:48.917234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-54cb30c5-9126-411f-a7d1-be575c9fa466 None None] MFA Rules not processed for user `13603c6cd9544f5391256ea600ad0709`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:48.922172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-54cb30c5-9126-411f-a7d1-be575c9fa466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:48.950841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:48.979342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:49.007776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b04f62-9efa-4de0-b7c0-8cbba85782b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:49.011685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b04f62-9efa-4de0-b7c0-8cbba85782b9 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:49.066988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 860/1702] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:49.073366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a4b5-55b8-4b26-8b14-bce4e56faf5e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:49.073659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a4b5-55b8-4b26-8b14-bce4e56faf5e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:49.073938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a4b5-55b8-4b26-8b14-bce4e56faf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:49.096664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 843/1703] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:34:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:49.174800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-dcf3a4b5-55b8-4b26-8b14-bce4e56faf5e None None] MFA Rules not processed for user `91ea94e38e384e9ea072d444c8e4bb26`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:49.183011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dcf3a4b5-55b8-4b26-8b14-bce4e56faf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:49.239181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb30c5-9126-411f-a7d1-be575c9fa466 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:49.241073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-54cb30c5-9126-411f-a7d1-be575c9fa466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:49.274912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb30c5-9126-411f-a7d1-be575c9fa466 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:49.289900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0caaa-27dc-4c07-b437-2c8b56448d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:49.290508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0caaa-27dc-4c07-b437-2c8b56448d92 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:49.290776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0caaa-27dc-4c07-b437-2c8b56448d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:49.293940 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0caaa-27dc-4c07-b437-2c8b56448d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:49.314309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0caaa-27dc-4c07-b437-2c8b56448d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:49.317321 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0caaa-27dc-4c07-b437-2c8b56448d92 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:49.339005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 844/1704] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:49.404662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 861/1705] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:49.415752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0caaa-27dc-4c07-b437-2c8b56448d92 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:49.572556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0caaa-27dc-4c07-b437-2c8b56448d92 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:49.581689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a10b4-e6b9-4f66-9021-d1f834b095f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:49.582442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a10b4-e6b9-4f66-9021-d1f834b095f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:49.583103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a10b4-e6b9-4f66-9021-d1f834b095f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:49.584626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a10b4-e6b9-4f66-9021-d1f834b095f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:49.609331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a10b4-e6b9-4f66-9021-d1f834b095f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:49.611144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a10b4-e6b9-4f66-9021-d1f834b095f9 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:49.637669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 845/1706] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:49.755838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a4b5-55b8-4b26-8b14-bce4e56faf5e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:49.777161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a10b4-e6b9-4f66-9021-d1f834b095f9 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:49.778485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a4b5-55b8-4b26-8b14-bce4e56faf5e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:49.783285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9648c64-5ae5-4ec4-a500-cefad993c3b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:49.784346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9648c64-5ae5-4ec4-a500-cefad993c3b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:49.784346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9648c64-5ae5-4ec4-a500-cefad993c3b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:49.784749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9648c64-5ae5-4ec4-a500-cefad993c3b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:49.915708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9648c64-5ae5-4ec4-a500-cefad993c3b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:49.917340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9648c64-5ae5-4ec4-a500-cefad993c3b2 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:49.934413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a10b4-e6b9-4f66-9021-d1f834b095f9 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:49.944486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4afbe231a9184f0788ecead87afd1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82c59067fd754d1aa176231ccd7fcc88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:49.946158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 862/1707] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:49.951964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:49.952474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:49.953063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:49.955279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:49.956816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:49.962465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:49.993533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 846/1708] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:49.995027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9648c64-5ae5-4ec4-a500-cefad993c3b2 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:50.014654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9648c64-5ae5-4ec4-a500-cefad993c3b2 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:50.034691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a396220-15d0-4751-89c8-072dfc9b00a9 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4afbe231a9184f0788ecead87afd1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82c59067fd754d1aa176231ccd7fcc88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:50.034855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a396220-15d0-4751-89c8-072dfc9b00a9 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:50.035152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a396220-15d0-4751-89c8-072dfc9b00a9 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:50.035431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a396220-15d0-4751-89c8-072dfc9b00a9 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:50.036447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a396220-15d0-4751-89c8-072dfc9b00a9 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:50.038505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a396220-15d0-4751-89c8-072dfc9b00a9 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:50.129453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 863/1709] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:34:49 2022] POST /identity/v3/projects => generated 353 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:34:50.132951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:50.149017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c958b4-8152-4058-b59d-023086e95c8f tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:50.372886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:50.397591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4afbe231a9184f0788ecead87afd1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82c59067fd754d1aa176231ccd7fcc88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:50.398261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:50.398561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:50.398896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:50.399745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:34:50.399903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:50.406078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:50.465055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 847/1710] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:34:50.548797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdce32-41af-4ccc-bfbb-e23229e82e15 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:50.549922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdce32-41af-4ccc-bfbb-e23229e82e15 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:50.550211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdce32-41af-4ccc-bfbb-e23229e82e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:50.623272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:50.641717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff769a9-c717-4b40-b983-aee648bafa57 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:50.648593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-64bdce32-41af-4ccc-bfbb-e23229e82e15 None None] MFA Rules not processed for user `13ac62bdb9bc4c7f94dad08f810559e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:50.651942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4afbe231a9184f0788ecead87afd1d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82c59067fd754d1aa176231ccd7fcc88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:50.652748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:50.653119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:50.653511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] PATH_INFO: `/v3/projects/58130648bd6c4b0985f60e4e5f7002b3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:50.655290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Authorizing `identity:delete_project(project_id=58130648bd6c4b0985f60e4e5f7002b3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:50.657416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:50.664837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-64bdce32-41af-4ccc-bfbb-e23229e82e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:50.796392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:50.933598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 864/1711] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:50.934254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58130648bd6c4b0985f60e4e5f7002b3', 'request_id': 'req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:50.951880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdce32-41af-4ccc-bfbb-e23229e82e15 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:50.953326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-64bdce32-41af-4ccc-bfbb-e23229e82e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:50.959612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58130648bd6c4b0985f60e4e5f7002b3', 'request_id': 'req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:50.993385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58130648bd6c4b0985f60e4e5f7002b3', 'request_id': 'req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:51.026992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] The token cache is being invalidate because project 58130648bd6c4b0985f60e4e5f7002b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:34:51.027303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:51.031010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:51.061598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 848/1712] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:34:50 2022] DELETE /identity/v3/projects/58130648bd6c4b0985f60e4e5f7002b3 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:51.062775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:51.070176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:51.211075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdce32-41af-4ccc-bfbb-e23229e82e15 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:51.334999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3b546-a64a-4782-9d8d-361ae2d75d8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:51.335837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3b546-a64a-4782-9d8d-361ae2d75d8b service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:51.336613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3b546-a64a-4782-9d8d-361ae2d75d8b service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:51.337544 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3b546-a64a-4782-9d8d-361ae2d75d8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:51.339762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-55e3b546-a64a-4782-9d8d-361ae2d75d8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:51.402914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f6291-deb1-400b-9b8e-7b2d71bfab60 tempest-VolumeQuotaClassesTest-402467991 tempest-VolumeQuotaClassesTest-402467991-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:51.424880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2c2df-7187-43ca-8c97-028023c3d04a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:51.425741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2c2df-7187-43ca-8c97-028023c3d04a service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:51.426202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2c2df-7187-43ca-8c97-028023c3d04a service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:51.426510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2c2df-7187-43ca-8c97-028023c3d04a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:51.431387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-01e2c2df-7187-43ca-8c97-028023c3d04a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:51.492131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2c2df-7187-43ca-8c97-028023c3d04a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:51.493346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2c2df-7187-43ca-8c97-028023c3d04a service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:51.533616 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 849/1713] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:51.534700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193a5bf-671c-46ff-aed3-f4790477829e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:51.540967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193a5bf-671c-46ff-aed3-f4790477829e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:51.542407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193a5bf-671c-46ff-aed3-f4790477829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:51.601988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3b546-a64a-4782-9d8d-361ae2d75d8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:51.603678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3b546-a64a-4782-9d8d-361ae2d75d8b service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:51.638041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 865/1714] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:34:51.639443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f9f8e-7042-43fd-820b-7535eb99dc39 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:51.639722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f9f8e-7042-43fd-820b-7535eb99dc39 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:51.639988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f9f8e-7042-43fd-820b-7535eb99dc39 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:51.643787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 866/1715] 10.222.0.42 () {54 vars in 962 bytes} [Wed Aug 31 04:34:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:34:51.645968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a193a5bf-671c-46ff-aed3-f4790477829e None None] MFA Rules not processed for user `863e457bd6fd4bff82f552e63ca1034c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:51.647235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a193a5bf-671c-46ff-aed3-f4790477829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:51.656430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9402-3e73-49ae-a335-5c023411baab None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:51.656430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9402-3e73-49ae-a335-5c023411baab None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:51.656430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9402-3e73-49ae-a335-5c023411baab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:51.719106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e11f9402-3e73-49ae-a335-5c023411baab None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:51.720350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e11f9402-3e73-49ae-a335-5c023411baab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:51.767877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 867/1716] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Aug 31 04:34:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:34:51.775236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649a04-d376-4171-a27d-c2b970c4d77a None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:51.776871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649a04-d376-4171-a27d-c2b970c4d77a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:51.777928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649a04-d376-4171-a27d-c2b970c4d77a None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:51.792790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 868/1717] 10.222.0.42 () {54 vars in 912 bytes} [Wed Aug 31 04:34:51 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:34:51.796440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b2b68-c847-4a53-aa48-0ceb6b51ad9f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:51.799888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b2b68-c847-4a53-aa48-0ceb6b51ad9f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:51.801015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b2b68-c847-4a53-aa48-0ceb6b51ad9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:51.850245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:51.880778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:51.881074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a92b2b68-c847-4a53-aa48-0ceb6b51ad9f None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:51.888636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a92b2b68-c847-4a53-aa48-0ceb6b51ad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:51.952943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 869/1718] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:34:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:51.997003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 850/1719] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:51.997527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd0fcf-b1b3-4fef-ae5f-f7c10af08e9d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:51.997637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd0fcf-b1b3-4fef-ae5f-f7c10af08e9d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:51.997901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd0fcf-b1b3-4fef-ae5f-f7c10af08e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:52.010040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193a5bf-671c-46ff-aed3-f4790477829e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:52.011917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a193a5bf-671c-46ff-aed3-f4790477829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:52.058505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-60fd0fcf-b1b3-4fef-ae5f-f7c10af08e9d None None] MFA Rules not processed for user `863e457bd6fd4bff82f552e63ca1034c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:52.068177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-60fd0fcf-b1b3-4fef-ae5f-f7c10af08e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:52.109021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 870/1720] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:34:52.114266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd0fcf-b1b3-4fef-ae5f-f7c10af08e9d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:52.116031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-60fd0fcf-b1b3-4fef-ae5f-f7c10af08e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:52.172383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193a5bf-671c-46ff-aed3-f4790477829e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:52.188914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:52.191183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:52.191856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:52.193380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:52.232691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:52.234484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:52.258481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 851/1721] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:34:52.266130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:52.270700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:52.335996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd0fcf-b1b3-4fef-ae5f-f7c10af08e9d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:52.347209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822c7f8-f5c9-4605-b5fa-cbf8c2c68396 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:52.365706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:52.366381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:52.366660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:52.366953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] PATH_INFO: `/v3/users/a4ea075c1b5e4c04bec5643dd44edc9a` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:52.368632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ea075c1b5e4c04bec5643dd44edc9a)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:52.372646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:52.378765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66d1c7-1bdf-44e2-8286-706e5130c6b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:52.379402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d1c7-1bdf-44e2-8286-706e5130c6b5 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:52.379813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d1c7-1bdf-44e2-8286-706e5130c6b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:52.380150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66d1c7-1bdf-44e2-8286-706e5130c6b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:52.398463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66d1c7-1bdf-44e2-8286-706e5130c6b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:52.402059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66d1c7-1bdf-44e2-8286-706e5130c6b5 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:52.451194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 852/1722] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:52.544636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ea075c1b5e4c04bec5643dd44edc9a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:52.567403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ea075c1b5e4c04bec5643dd44edc9a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:52.568651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ea075c1b5e4c04bec5643dd44edc9a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:52.613084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ea075c1b5e4c04bec5643dd44edc9a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:52.614578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ea075c1b5e4c04bec5643dd44edc9a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:52.643911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 871/1723] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:34:52 2022] DELETE /identity/v3/users/a4ea075c1b5e4c04bec5643dd44edc9a => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:52.648555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51fe22-79c5-4da5-ba37-c3765f5fda59 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:52.648919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51fe22-79c5-4da5-ba37-c3765f5fda59 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:52.649265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51fe22-79c5-4da5-ba37-c3765f5fda59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:52.706757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bc51fe22-79c5-4da5-ba37-c3765f5fda59 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:52.716716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc51fe22-79c5-4da5-ba37-c3765f5fda59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:52.927427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 853/1724] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:34:52.987325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:52.989081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:53.248974 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a80a0a-3eeb-4878-a675-f3020cdb8574 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:53.262629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:53.265339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:53.265339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:53.265339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:53.309284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:53.310809 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:53.362158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 872/1725] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:53.828651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7d48e-4247-4582-9a53-6870021d5fd7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:53.829630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7d48e-4247-4582-9a53-6870021d5fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:53.830039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7d48e-4247-4582-9a53-6870021d5fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:53.875927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-04d7d48e-4247-4582-9a53-6870021d5fd7 None None] MFA Rules not processed for user `718418f954f94dad93eeb725e345c850`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:53.877285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-04d7d48e-4247-4582-9a53-6870021d5fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:54.012987 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 854/1726] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:54.124377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:54.125794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:54.147014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:54.224127 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdda23-2ffb-4e2a-9777-a208aac62019 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:54.234120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbdce8-6fb3-419f-a7a6-f56d4176d5bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:54.235072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbdce8-6fb3-419f-a7a6-f56d4176d5bd service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:54.235539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbdce8-6fb3-419f-a7a6-f56d4176d5bd service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:54.235870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbdce8-6fb3-419f-a7a6-f56d4176d5bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:54.247719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbdce8-6fb3-419f-a7a6-f56d4176d5bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:54.250091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbdce8-6fb3-419f-a7a6-f56d4176d5bd service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:54.264793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 873/1727] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:54.733539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7d48e-4247-4582-9a53-6870021d5fd7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:54.735090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-04d7d48e-4247-4582-9a53-6870021d5fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:54.761390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7d48e-4247-4582-9a53-6870021d5fd7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:54.779844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:54.780513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:54.780795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:54.781063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] PATH_INFO: `/v3/projects/9f1ea53eaafb4650867378af0b21bc4f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:54.782405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1ea53eaafb4650867378af0b21bc4f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:54.784495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:54.786937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:55.020605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1ea53eaafb4650867378af0b21bc4f', 'request_id': 'req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.087387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da93ddf-5a2c-4141-9019-bb6a0b6f0143 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:55.087918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da93ddf-5a2c-4141-9019-bb6a0b6f0143 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:55.088354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da93ddf-5a2c-4141-9019-bb6a0b6f0143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:55.095001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1ea53eaafb4650867378af0b21bc4f', 'request_id': 'req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.134623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1ea53eaafb4650867378af0b21bc4f', 'request_id': 'req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.154443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9da93ddf-5a2c-4141-9019-bb6a0b6f0143 None None] MFA Rules not processed for user `863e457bd6fd4bff82f552e63ca1034c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:55.164083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9da93ddf-5a2c-4141-9019-bb6a0b6f0143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:55.184151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] The token cache is being invalidate because project 9f1ea53eaafb4650867378af0b21bc4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:34:55.184509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.187748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.190626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 855/1728] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:34:54 2022] DELETE /identity/v3/projects/9f1ea53eaafb4650867378af0b21bc4f => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:55.195373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:55.198128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:55.352009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 874/1729] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:34:55.418830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5778bd-de3f-4e69-90e6-9f0bda3b6859 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:55.433999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:55.434874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:55.435296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:55.435705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] PATH_INFO: `/v3/users/4afbe231a9184f0788ecead87afd1d68` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:55.438406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4afbe231a9184f0788ecead87afd1d68)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:55.440634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:55.472301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3c60-9a3a-4719-9a04-e989e9015513 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:55.472592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3c60-9a3a-4719-9a04-e989e9015513 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:55.472870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3c60-9a3a-4719-9a04-e989e9015513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:55.653700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4afbe231a9184f0788ecead87afd1d68'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.685834 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4afbe231a9184f0788ecead87afd1d68'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.687078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4afbe231a9184f0788ecead87afd1d68'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.713120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bdfc3c60-9a3a-4719-9a04-e989e9015513 None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:55.714394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bdfc3c60-9a3a-4719-9a04-e989e9015513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:55.754092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4afbe231a9184f0788ecead87afd1d68'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.754912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4afbe231a9184f0788ecead87afd1d68'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:55.803512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 856/1730] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:34:55 2022] DELETE /identity/v3/users/4afbe231a9184f0788ecead87afd1d68 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:55.984749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 875/1731] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:34:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:56.434039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:56.436394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:56.473307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:56.778587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b23e9-eff8-46d6-b240-140555467e94 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:56.790761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570df386-2a7e-49f9-aed1-3d76c3712037 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:56.791787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df386-2a7e-49f9-aed1-3d76c3712037 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:56.791920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df386-2a7e-49f9-aed1-3d76c3712037 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:56.792361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df386-2a7e-49f9-aed1-3d76c3712037 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:56.868788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3c60-9a3a-4719-9a04-e989e9015513 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:56.870913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bdfc3c60-9a3a-4719-9a04-e989e9015513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:56.905663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3c60-9a3a-4719-9a04-e989e9015513 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:56.961992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:56.991898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570df386-2a7e-49f9-aed1-3d76c3712037 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:56.995605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570df386-2a7e-49f9-aed1-3d76c3712037 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:57.028451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 857/1732] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:34:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 04:34:57.191593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:57.192527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:57.195485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:57.195763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] PATH_INFO: `/v3/users/13ff7867572c48889bba01106b771765` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:57.203652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ff7867572c48889bba01106b771765)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:57.213162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:57.388989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ff7867572c48889bba01106b771765'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:57.409738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ff7867572c48889bba01106b771765'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:57.410404 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ff7867572c48889bba01106b771765'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:57.448141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ff7867572c48889bba01106b771765'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:57.450517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ff7867572c48889bba01106b771765'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:57.502688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 876/1733] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:34:56 2022] DELETE /identity/v3/users/13ff7867572c48889bba01106b771765 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:57.504422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504623da-5fd1-487e-897d-2247749c871b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:57.504895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504623da-5fd1-487e-897d-2247749c871b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:57.505124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504623da-5fd1-487e-897d-2247749c871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:57.571085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-504623da-5fd1-487e-897d-2247749c871b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:57.579281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-504623da-5fd1-487e-897d-2247749c871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:57.736110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 858/1734] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:57.752865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:57.755665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:57.967717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08af0da-b785-427e-94d4-3062db894d6c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:57.978298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:57.979058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:57.979534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:57.979976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:58.000733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:58.004184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:58.030632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 877/1735] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:58.162953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a15c0-6988-4424-ae46-4aa5b6b5addd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:58.163405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a15c0-6988-4424-ae46-4aa5b6b5addd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:58.163800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a15c0-6988-4424-ae46-4aa5b6b5addd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:58.272961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4e9a15c0-6988-4424-ae46-4aa5b6b5addd None None] MFA Rules not processed for user `13603c6cd9544f5391256ea600ad0709`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:58.274210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e9a15c0-6988-4424-ae46-4aa5b6b5addd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:58.324385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 859/1736] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:34:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:58.375964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:58.377859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:58.402657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04474fcc-e276-492c-abca-8cea50750f67 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:58.416206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29216f0-03d2-44ff-8bb8-289620fc4f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:58.417151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29216f0-03d2-44ff-8bb8-289620fc4f04 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:58.420037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29216f0-03d2-44ff-8bb8-289620fc4f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:58.420507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29216f0-03d2-44ff-8bb8-289620fc4f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:58.445300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29216f0-03d2-44ff-8bb8-289620fc4f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:58.451772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29216f0-03d2-44ff-8bb8-289620fc4f04 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:58.500998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 878/1737] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:34:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:34:59.281361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a15c0-6988-4424-ae46-4aa5b6b5addd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:34:59.283044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e9a15c0-6988-4424-ae46-4aa5b6b5addd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:59.355850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a15c0-6988-4424-ae46-4aa5b6b5addd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:34:59.378730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:34:59.379391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:59.379667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:59.379941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] PATH_INFO: `/v3/projects/82c59067fd754d1aa176231ccd7fcc88` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:59.385974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c59067fd754d1aa176231ccd7fcc88)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:34:59.394822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:34:59.503413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:59.595261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac262ca5-8a7a-488a-ac1a-f69ab854269c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:59.596181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac262ca5-8a7a-488a-ac1a-f69ab854269c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:59.596646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac262ca5-8a7a-488a-ac1a-f69ab854269c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:59.624459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c59067fd754d1aa176231ccd7fcc88', 'request_id': 'req-3283a194-46c1-40f3-b00c-2375fc36c164'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:59.655609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c59067fd754d1aa176231ccd7fcc88', 'request_id': 'req-3283a194-46c1-40f3-b00c-2375fc36c164'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:59.682557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ac262ca5-8a7a-488a-ac1a-f69ab854269c None None] MFA Rules not processed for user `13603c6cd9544f5391256ea600ad0709`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:59.700060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ac262ca5-8a7a-488a-ac1a-f69ab854269c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:59.700873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c59067fd754d1aa176231ccd7fcc88', 'request_id': 'req-3283a194-46c1-40f3-b00c-2375fc36c164'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:59.800473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] The token cache is being invalidate because project 82c59067fd754d1aa176231ccd7fcc88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:34:59.800768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:59.802377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3283a194-46c1-40f3-b00c-2375fc36c164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:34:59.812428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 860/1738] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:34:59 2022] DELETE /identity/v3/projects/82c59067fd754d1aa176231ccd7fcc88 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:34:59.819498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75933b-27f0-4288-a546-ab7974eded07 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:34:59.819941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75933b-27f0-4288-a546-ab7974eded07 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:34:59.820351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75933b-27f0-4288-a546-ab7974eded07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:34:59.840362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:34:59.910074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9c75933b-27f0-4288-a546-ab7974eded07 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:34:59.915578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9c75933b-27f0-4288-a546-ab7974eded07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:34:59.995148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 879/1739] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 31 04:34:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:34:59.998618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8145a-cbc9-4190-a27c-941128e36984 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:00.001478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8145a-cbc9-4190-a27c-941128e36984 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:00.002327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8145a-cbc9-4190-a27c-941128e36984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:00.133345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9fb8145a-cbc9-4190-a27c-941128e36984 None None] MFA Rules not processed for user `13ac62bdb9bc4c7f94dad08f810559e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:00.138086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9fb8145a-cbc9-4190-a27c-941128e36984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:00.226450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 861/1740] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:00.249998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75933b-27f0-4288-a546-ab7974eded07 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:00.262922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9c75933b-27f0-4288-a546-ab7974eded07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:00.292992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75933b-27f0-4288-a546-ab7974eded07 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:00.313562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:00.316498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:00.316963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:00.317396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] PATH_INFO: `/v3/projects/51854995213c448882c666ef1cbe2579` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:00.327348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=51854995213c448882c666ef1cbe2579)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:00.334976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:00.432125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 880/1741] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:34:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:00.434365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8145a-cbc9-4190-a27c-941128e36984 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:00.445176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9fb8145a-cbc9-4190-a27c-941128e36984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:00.514084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8145a-cbc9-4190-a27c-941128e36984 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:00.521248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51854995213c448882c666ef1cbe2579', 'request_id': 'req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:00.557891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51854995213c448882c666ef1cbe2579', 'request_id': 'req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:00.581876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51854995213c448882c666ef1cbe2579', 'request_id': 'req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:00.625146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] The token cache is being invalidate because project 51854995213c448882c666ef1cbe2579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:00.625439 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:00.630805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3ed356df-4506-4f55-96b4-d3a03d8fd4fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:00.635137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 862/1742] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:00 2022] DELETE /identity/v3/projects/51854995213c448882c666ef1cbe2579 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:00.642352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5acc5-f2de-40f5-8c3f-71903e0e41d7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:00.643037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5acc5-f2de-40f5-8c3f-71903e0e41d7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:00.643511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5acc5-f2de-40f5-8c3f-71903e0e41d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:00.694138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-23b5acc5-f2de-40f5-8c3f-71903e0e41d7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:00.696692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:00.698585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-23b5acc5-f2de-40f5-8c3f-71903e0e41d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:00.699407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:00.699854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:00.700264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:00.704620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:00.761705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:00.772894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:00.800188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 881/1743] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:00.803678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:00.817088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:00.822105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:00.906785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 863/1744] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:00.909335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5acc5-f2de-40f5-8c3f-71903e0e41d7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:00.939344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5acc5-f2de-40f5-8c3f-71903e0e41d7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:00.952620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:00.953600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:00.953970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:00.954327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:00.954903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:00.955223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:00.965851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:01.013987 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 864/1745] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:35:01.017951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:01.035989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc53982-2384-4b2a-a26d-c609f2b08934 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:01.055795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7c58c-9b24-4959-9e45-3af9afc301b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:01.058672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7c58c-9b24-4959-9e45-3af9afc301b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:01.059084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7c58c-9b24-4959-9e45-3af9afc301b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:01.059703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7c58c-9b24-4959-9e45-3af9afc301b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:01.060028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7c58c-9b24-4959-9e45-3af9afc301b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:01.070303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7c58c-9b24-4959-9e45-3af9afc301b8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:01.111089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23566bc0-2138-4982-bae6-92860a9ff27a service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:01.131764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:01.132474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:01.132824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:01.133115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] PATH_INFO: `/v3/users/91ea94e38e384e9ea072d444c8e4bb26` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:01.139665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ea94e38e384e9ea072d444c8e4bb26)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:01.141642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:01.180350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 865/1746] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:01 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:01.186818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7c58c-9b24-4959-9e45-3af9afc301b8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:01.244637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:01.345394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ea94e38e384e9ea072d444c8e4bb26'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:01.371970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ea94e38e384e9ea072d444c8e4bb26'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:01.372952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ea94e38e384e9ea072d444c8e4bb26'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:01.434559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7c58c-9b24-4959-9e45-3af9afc301b8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:01.444411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ea94e38e384e9ea072d444c8e4bb26'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:01.445360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d706e182-1cd0-44a2-9234-decb3b0ce780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ea94e38e384e9ea072d444c8e4bb26'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:01.453490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:01.454878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:01.455409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:01.456144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:01.457035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:01.460433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:01.494214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 882/1747] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:00 2022] DELETE /identity/v3/users/91ea94e38e384e9ea072d444c8e4bb26 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:01.497415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b294-d4b0-46f4-835a-b90b8b1cc9e7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:01.497705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b294-d4b0-46f4-835a-b90b8b1cc9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:01.497978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470b294-d4b0-46f4-835a-b90b8b1cc9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:01.510961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 866/1748] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:01 2022] POST /identity/v3/users => generated 287 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:01.512930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:01.515634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:01.564546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b470b294-d4b0-46f4-835a-b90b8b1cc9e7 None None] MFA Rules not processed for user `718418f954f94dad93eeb725e345c850`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:01.564914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa023e9-c95e-4c68-9845-71af1de62a04 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:01.566074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b470b294-d4b0-46f4-835a-b90b8b1cc9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:01.702484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 883/1749] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:01.792870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f42779-b22c-4ed1-806b-407e25222ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:01.794853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f42779-b22c-4ed1-806b-407e25222ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:01.795874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f42779-b22c-4ed1-806b-407e25222ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:01.796703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f42779-b22c-4ed1-806b-407e25222ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:01.799257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f42779-b22c-4ed1-806b-407e25222ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:01.815736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f42779-b22c-4ed1-806b-407e25222ae5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:01.856246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 867/1750] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:01.860061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470b294-d4b0-46f4-835a-b90b8b1cc9e7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:01.876363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470b294-d4b0-46f4-835a-b90b8b1cc9e7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:01.891921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb0764-88fc-481a-80b6-6f51588e688a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:01.892929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb0764-88fc-481a-80b6-6f51588e688a admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:01.893401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb0764-88fc-481a-80b6-6f51588e688a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:01.897323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb0764-88fc-481a-80b6-6f51588e688a admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:01.898663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb0764-88fc-481a-80b6-6f51588e688a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:01.901148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb0764-88fc-481a-80b6-6f51588e688a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:01.957203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 884/1751] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:01.960589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f42779-b22c-4ed1-806b-407e25222ae5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:01.991417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f42779-b22c-4ed1-806b-407e25222ae5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:02.012372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:02.017502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:02.017795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:02.018072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] PATH_INFO: `/v3/projects/365e83c8647b401da0040ff09e9cda7e/users/6a11c5a47a7d48528916c1fc6a6e681e/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:02.108861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365e83c8647b401da0040ff09e9cda7e, user_id=6a11c5a47a7d48528916c1fc6a6e681e, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:02.120033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:02.139325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:02.160055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 868/1752] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:01 2022] PUT /identity/v3/projects/365e83c8647b401da0040ff09e9cda7e/users/6a11c5a47a7d48528916c1fc6a6e681e/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:02.165464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765be66-af75-4da1-8759-f27279da82b2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:02.166612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765be66-af75-4da1-8759-f27279da82b2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:02.166939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765be66-af75-4da1-8759-f27279da82b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:02.197807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6765be66-af75-4da1-8759-f27279da82b2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:02.199393 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6765be66-af75-4da1-8759-f27279da82b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:02.380229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 885/1753] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:02.441692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:02.446640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:02.651166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d5ab7-791b-463a-9f8c-c57b0d7d6923 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:02.665277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224fab6-e470-409b-8e46-a88f51929c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:02.667452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224fab6-e470-409b-8e46-a88f51929c8e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:02.667861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224fab6-e470-409b-8e46-a88f51929c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:02.669266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224fab6-e470-409b-8e46-a88f51929c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:02.702318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224fab6-e470-409b-8e46-a88f51929c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:02.705888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224fab6-e470-409b-8e46-a88f51929c8e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:02.746280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 869/1754] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:03.549677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765be66-af75-4da1-8759-f27279da82b2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:03.554777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6765be66-af75-4da1-8759-f27279da82b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:03.585893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765be66-af75-4da1-8759-f27279da82b2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:03.605177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:03.606200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:03.606546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:03.607071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] PATH_INFO: `/v3/projects/e7828485ce8545c7985c966188fd1349` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:03.609604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7828485ce8545c7985c966188fd1349)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:03.614472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:03.719548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:03.768954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7828485ce8545c7985c966188fd1349', 'request_id': 'req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:03.800257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7828485ce8545c7985c966188fd1349', 'request_id': 'req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:03.816041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7828485ce8545c7985c966188fd1349', 'request_id': 'req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:03.865745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] The token cache is being invalidate because project e7828485ce8545c7985c966188fd1349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:03.866733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:03.870350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-34c5cba1-7b9b-4732-a7aa-a8137a03ef96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:03.883515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 886/1755] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:03 2022] DELETE /identity/v3/projects/e7828485ce8545c7985c966188fd1349 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:03.887202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b59a73-e38e-480c-9ba1-b82aaea2a77e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:03.887202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b59a73-e38e-480c-9ba1-b82aaea2a77e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:03.887202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b59a73-e38e-480c-9ba1-b82aaea2a77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:03.930874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-76b59a73-e38e-480c-9ba1-b82aaea2a77e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:03.933784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-76b59a73-e38e-480c-9ba1-b82aaea2a77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:03.972874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29ed95-718e-44ea-96d6-13a709adedb6 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:03.973169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29ed95-718e-44ea-96d6-13a709adedb6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:03.973445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29ed95-718e-44ea-96d6-13a709adedb6 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:03.975764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 887/1756] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:35:04.047807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 870/1757] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:04.056587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ccdd6-72b5-41b8-915a-b1bf67510393 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.056877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ccdd6-72b5-41b8-915a-b1bf67510393 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.057158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ccdd6-72b5-41b8-915a-b1bf67510393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.086990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-743ccdd6-72b5-41b8-915a-b1bf67510393 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:04.088339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-743ccdd6-72b5-41b8-915a-b1bf67510393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:04.098970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 888/1758] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:04.103493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b59a73-e38e-480c-9ba1-b82aaea2a77e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:04.119769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b59a73-e38e-480c-9ba1-b82aaea2a77e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:04.125428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:04.126159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.126541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.126904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.127675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:04.128042 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:04.129870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:04.151751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 871/1759] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:35:04.156301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ccdd6-72b5-41b8-915a-b1bf67510393 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:04.170111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ccdd6-72b5-41b8-915a-b1bf67510393 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:04.178422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ed282-28a1-4469-b4f0-06f165ec2472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:04.179266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ed282-28a1-4469-b4f0-06f165ec2472 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.179600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ed282-28a1-4469-b4f0-06f165ec2472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.179879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ed282-28a1-4469-b4f0-06f165ec2472 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.181150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ed282-28a1-4469-b4f0-06f165ec2472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:04.183545 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ed282-28a1-4469-b4f0-06f165ec2472 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:04.236082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 889/1760] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:04 2022] POST /identity/v3/projects => generated 391 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:04.243038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:04.249592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2ce3b-b4c2-47f1-95ed-82842db4892d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.249883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2ce3b-b4c2-47f1-95ed-82842db4892d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.250139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2ce3b-b4c2-47f1-95ed-82842db4892d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.282654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73289cbe-0470-4fe8-aaf0-abd81a4bebb4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:04.285961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8ba2ce3b-b4c2-47f1-95ed-82842db4892d None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:04.287680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8ba2ce3b-b4c2-47f1-95ed-82842db4892d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:04.413559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 890/1761] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:35:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:04.421505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2ce3b-b4c2-47f1-95ed-82842db4892d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:04.423183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8ba2ce3b-b4c2-47f1-95ed-82842db4892d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:04.442186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2ce3b-b4c2-47f1-95ed-82842db4892d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:04.448626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a39c-0f5c-48c3-b96b-5e47e26f5067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:04.449666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a39c-0f5c-48c3-b96b-5e47e26f5067 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.449666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a39c-0f5c-48c3-b96b-5e47e26f5067 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.449854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751a39c-0f5c-48c3-b96b-5e47e26f5067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.462117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751a39c-0f5c-48c3-b96b-5e47e26f5067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:04.464574 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751a39c-0f5c-48c3-b96b-5e47e26f5067 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:04.469984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300b9ba-eaa7-4c81-b4d5-584867b1dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:04.471127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300b9ba-eaa7-4c81-b4d5-584867b1dc25 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.471639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300b9ba-eaa7-4c81-b4d5-584867b1dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.472298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300b9ba-eaa7-4c81-b4d5-584867b1dc25 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.473301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300b9ba-eaa7-4c81-b4d5-584867b1dc25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:04.477091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300b9ba-eaa7-4c81-b4d5-584867b1dc25 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:04.478282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 891/1762] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:04.528047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 872/1763] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:04 2022] POST /identity/v3/users => generated 296 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:04.532600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a39c-0f5c-48c3-b96b-5e47e26f5067 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:04.545096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751a39c-0f5c-48c3-b96b-5e47e26f5067 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:04.550910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94d63e-1717-4a06-8395-f22df544599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:04.552197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d63e-1717-4a06-8395-f22df544599f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.553073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d63e-1717-4a06-8395-f22df544599f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.553835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d63e-1717-4a06-8395-f22df544599f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.555133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94d63e-1717-4a06-8395-f22df544599f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:04.557244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94d63e-1717-4a06-8395-f22df544599f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:04.572149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 892/1764] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:04.577938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300b9ba-eaa7-4c81-b4d5-584867b1dc25 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:04.590880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300b9ba-eaa7-4c81-b4d5-584867b1dc25 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:04.596877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1256a8b-db0c-4733-865e-e51886737092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:04.597610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256a8b-db0c-4733-865e-e51886737092 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.597954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256a8b-db0c-4733-865e-e51886737092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.598226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256a8b-db0c-4733-865e-e51886737092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.599032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1256a8b-db0c-4733-865e-e51886737092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:04.600858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1256a8b-db0c-4733-865e-e51886737092 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:04.612697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 873/1765] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:04.617338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94d63e-1717-4a06-8395-f22df544599f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:04.630349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94d63e-1717-4a06-8395-f22df544599f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:04.636721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35727ddf-7844-44ff-9cd5-5563d2040e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:04.637708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35727ddf-7844-44ff-9cd5-5563d2040e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.637990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35727ddf-7844-44ff-9cd5-5563d2040e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.638257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35727ddf-7844-44ff-9cd5-5563d2040e63 admin admin] PATH_INFO: `/v3/projects/def0b40d176a4213b5e9c0e7a6778c02/users/0168d3a277a547f09f08a64c090c3191/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.654463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35727ddf-7844-44ff-9cd5-5563d2040e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=def0b40d176a4213b5e9c0e7a6778c02, user_id=0168d3a277a547f09f08a64c090c3191, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:04.657215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35727ddf-7844-44ff-9cd5-5563d2040e63 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:04.676006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 893/1766] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:04 2022] PUT /identity/v3/projects/def0b40d176a4213b5e9c0e7a6778c02/users/0168d3a277a547f09f08a64c090c3191/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:04.688675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69c934-8880-4e4a-ad72-e212a1093e0a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.689106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69c934-8880-4e4a-ad72-e212a1093e0a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.689438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69c934-8880-4e4a-ad72-e212a1093e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.709258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:04.736583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9f69c934-8880-4e4a-ad72-e212a1093e0a None None] MFA Rules not processed for user `0168d3a277a547f09f08a64c090c3191`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:04.739149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f69c934-8880-4e4a-ad72-e212a1093e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:04.803717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61977dda-0aa2-4c6f-b89f-0f31a1d7e3eb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.804005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61977dda-0aa2-4c6f-b89f-0f31a1d7e3eb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.804303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61977dda-0aa2-4c6f-b89f-0f31a1d7e3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:04.868337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 874/1767] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:04.869563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69c934-8880-4e4a-ad72-e212a1093e0a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:04.904204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-61977dda-0aa2-4c6f-b89f-0f31a1d7e3eb None None] MFA Rules not processed for user `6a11c5a47a7d48528916c1fc6a6e681e`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:04.905487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-61977dda-0aa2-4c6f-b89f-0f31a1d7e3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:04.948957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:04.981986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69c934-8880-4e4a-ad72-e212a1093e0a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:04.996322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab771c-53ba-44af-a922-0b9ed4a0b858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:04.998117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab771c-53ba-44af-a922-0b9ed4a0b858 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:04.998410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab771c-53ba-44af-a922-0b9ed4a0b858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:04.998690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab771c-53ba-44af-a922-0b9ed4a0b858 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.001543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab771c-53ba-44af-a922-0b9ed4a0b858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.007867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab771c-53ba-44af-a922-0b9ed4a0b858 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.078169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 894/1768] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:05.081538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61977dda-0aa2-4c6f-b89f-0f31a1d7e3eb None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.089792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-61977dda-0aa2-4c6f-b89f-0f31a1d7e3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:05.112387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 875/1769] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:04 2022] POST /identity/v3/projects => generated 393 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:05.120317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab771c-53ba-44af-a922-0b9ed4a0b858 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.142013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61977dda-0aa2-4c6f-b89f-0f31a1d7e3eb None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.292100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab771c-53ba-44af-a922-0b9ed4a0b858 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.300356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9311d0f-5683-4c25-aa67-fe60a75197bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:05.300974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9311d0f-5683-4c25-aa67-fe60a75197bc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.301255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9311d0f-5683-4c25-aa67-fe60a75197bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.301564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9311d0f-5683-4c25-aa67-fe60a75197bc admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.302212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9311d0f-5683-4c25-aa67-fe60a75197bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.304402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9311d0f-5683-4c25-aa67-fe60a75197bc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.326781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e89e-5f87-4651-865c-f2e37affa40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:05.327609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e89e-5f87-4651-865c-f2e37affa40e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.327902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e89e-5f87-4651-865c-f2e37affa40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.328261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e89e-5f87-4651-865c-f2e37affa40e admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.329286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e89e-5f87-4651-865c-f2e37affa40e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.331424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e89e-5f87-4651-865c-f2e37affa40e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.342643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 876/1770] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:05 2022] POST /identity/v3/users => generated 303 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:05.346689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9311d0f-5683-4c25-aa67-fe60a75197bc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.360528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9311d0f-5683-4c25-aa67-fe60a75197bc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.367120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c053fd-1e55-4a89-9661-bf8cc100d614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:05.367753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c053fd-1e55-4a89-9661-bf8cc100d614 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.368032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c053fd-1e55-4a89-9661-bf8cc100d614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.368601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c053fd-1e55-4a89-9661-bf8cc100d614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.368956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c053fd-1e55-4a89-9661-bf8cc100d614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.372691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c053fd-1e55-4a89-9661-bf8cc100d614 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.376931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 895/1771] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:05 2022] POST /identity/v3/projects => generated 373 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:05.382165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:05.383570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e89e-5f87-4651-865c-f2e37affa40e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.390645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 877/1772] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:35:05.391980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c053fd-1e55-4a89-9661-bf8cc100d614 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.405457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c053fd-1e55-4a89-9661-bf8cc100d614 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.497499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aa717-d421-4531-bd72-068628fb9b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:05.498619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aa717-d421-4531-bd72-068628fb9b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.499394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aa717-d421-4531-bd72-068628fb9b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.499909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aa717-d421-4531-bd72-068628fb9b79 admin admin] PATH_INFO: `/v3/projects/b295f1b0df2f4dd2a057502fa1599b91/users/4b6e4fc6c8d444609a5805c4a730c63f/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.530122 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20aa717-d421-4531-bd72-068628fb9b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b295f1b0df2f4dd2a057502fa1599b91, user_id=4b6e4fc6c8d444609a5805c4a730c63f, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.538600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20aa717-d421-4531-bd72-068628fb9b79 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.573171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 878/1773] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:05 2022] PUT /identity/v3/projects/b295f1b0df2f4dd2a057502fa1599b91/users/4b6e4fc6c8d444609a5805c4a730c63f/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:05.574200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aa717-d421-4531-bd72-068628fb9b79 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.578507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e89e-5f87-4651-865c-f2e37affa40e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.681065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:05.691275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0664a-8b01-46c8-9f05-a438b6f2da8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:05.692548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0664a-8b01-46c8-9f05-a438b6f2da8f admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.693315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0664a-8b01-46c8-9f05-a438b6f2da8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.694422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0664a-8b01-46c8-9f05-a438b6f2da8f admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.697101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0664a-8b01-46c8-9f05-a438b6f2da8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.701755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0664a-8b01-46c8-9f05-a438b6f2da8f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.722218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aa717-d421-4531-bd72-068628fb9b79 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.741514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612acaa-1bee-457b-a67c-6c77a0413c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:05.742442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612acaa-1bee-457b-a67c-6c77a0413c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.750105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612acaa-1bee-457b-a67c-6c77a0413c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.751295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612acaa-1bee-457b-a67c-6c77a0413c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.753163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612acaa-1bee-457b-a67c-6c77a0413c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.758126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612acaa-1bee-457b-a67c-6c77a0413c0e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.759531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 896/1774] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:05 2022] POST /identity/v3/users => generated 293 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:05.764169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0664a-8b01-46c8-9f05-a438b6f2da8f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.794265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 879/1775] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:05.800128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0664a-8b01-46c8-9f05-a438b6f2da8f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.801309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612acaa-1bee-457b-a67c-6c77a0413c0e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.813426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479c207-923e-4788-a06c-6aba37513019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:05.815316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479c207-923e-4788-a06c-6aba37513019 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.815907 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479c207-923e-4788-a06c-6aba37513019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.817691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479c207-923e-4788-a06c-6aba37513019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.818931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479c207-923e-4788-a06c-6aba37513019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.825359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612acaa-1bee-457b-a67c-6c77a0413c0e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.826276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479c207-923e-4788-a06c-6aba37513019 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.848974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7f8e4-3245-4db9-8476-974897aaebdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:05.850322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7f8e4-3245-4db9-8476-974897aaebdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.850889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7f8e4-3245-4db9-8476-974897aaebdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.851460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7f8e4-3245-4db9-8476-974897aaebdc admin admin] PATH_INFO: `/v3/projects/b295f1b0df2f4dd2a057502fa1599b91/users/4b6e4fc6c8d444609a5805c4a730c63f/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:05.853148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 897/1776] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:05.856513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7f8e4-3245-4db9-8476-974897aaebdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b295f1b0df2f4dd2a057502fa1599b91, user_id=4b6e4fc6c8d444609a5805c4a730c63f, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:05.864554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479c207-923e-4788-a06c-6aba37513019 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:05.867269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7f8e4-3245-4db9-8476-974897aaebdc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:05.890707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479c207-923e-4788-a06c-6aba37513019 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:05.905133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 880/1777] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:05 2022] PUT /identity/v3/projects/b295f1b0df2f4dd2a057502fa1599b91/users/4b6e4fc6c8d444609a5805c4a730c63f/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:05.919825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf46d53-a50c-4f93-a79e-bcc77c3c0e36 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:05.922525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf46d53-a50c-4f93-a79e-bcc77c3c0e36 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:05.923149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf46d53-a50c-4f93-a79e-bcc77c3c0e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:06.005042 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7cf46d53-a50c-4f93-a79e-bcc77c3c0e36 None None] MFA Rules not processed for user `4b6e4fc6c8d444609a5805c4a730c63f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:06.007675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7cf46d53-a50c-4f93-a79e-bcc77c3c0e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:06.069440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3da43-19ca-4665-b1fd-8ffc53a96d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:06.076601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3da43-19ca-4665-b1fd-8ffc53a96d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:06.077689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3da43-19ca-4665-b1fd-8ffc53a96d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:06.078167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3da43-19ca-4665-b1fd-8ffc53a96d08 admin admin] PATH_INFO: `/v3/projects/ac87c2498ad54abe8eadd4539055086b/users/2f123090eba448aca2d02c2111f80579/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:06.126866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3da43-19ca-4665-b1fd-8ffc53a96d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac87c2498ad54abe8eadd4539055086b, user_id=2f123090eba448aca2d02c2111f80579, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:06.129902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3da43-19ca-4665-b1fd-8ffc53a96d08 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:06.155687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:06.167434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 898/1778] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:05 2022] PUT /identity/v3/projects/ac87c2498ad54abe8eadd4539055086b/users/2f123090eba448aca2d02c2111f80579/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:06.171147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3da43-19ca-4665-b1fd-8ffc53a96d08 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:06.225879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 881/1779] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:05 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:06.244252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf46d53-a50c-4f93-a79e-bcc77c3c0e36 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:06.246267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7cf46d53-a50c-4f93-a79e-bcc77c3c0e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:06.369538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3da43-19ca-4665-b1fd-8ffc53a96d08 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:06.406318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf46d53-a50c-4f93-a79e-bcc77c3c0e36 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:06.417406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01950d29-1157-486f-b54e-291f6934bf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:06.418046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950d29-1157-486f-b54e-291f6934bf2c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:06.418323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950d29-1157-486f-b54e-291f6934bf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:06.423261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950d29-1157-486f-b54e-291f6934bf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:06.424045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01950d29-1157-486f-b54e-291f6934bf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:06.433686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01950d29-1157-486f-b54e-291f6934bf2c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:06.478987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 899/1780] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:06.488280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01950d29-1157-486f-b54e-291f6934bf2c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:06.488483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f8173-ed2a-43d5-a23e-d7d51b772b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:06.489275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f8173-ed2a-43d5-a23e-d7d51b772b3a service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:06.490732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f8173-ed2a-43d5-a23e-d7d51b772b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:06.491031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f8173-ed2a-43d5-a23e-d7d51b772b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:06.545230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01950d29-1157-486f-b54e-291f6934bf2c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:06.576772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a98279-2b53-4a77-a6b6-803115a18a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:06.577475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a98279-2b53-4a77-a6b6-803115a18a67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:06.577759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a98279-2b53-4a77-a6b6-803115a18a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:06.578031 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a98279-2b53-4a77-a6b6-803115a18a67 admin admin] PATH_INFO: `/v3/projects/ac87c2498ad54abe8eadd4539055086b/users/2f123090eba448aca2d02c2111f80579/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:06.586023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a98279-2b53-4a77-a6b6-803115a18a67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac87c2498ad54abe8eadd4539055086b, user_id=2f123090eba448aca2d02c2111f80579, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:06.593642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a98279-2b53-4a77-a6b6-803115a18a67 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:06.634855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 900/1781] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:06 2022] PUT /identity/v3/projects/ac87c2498ad54abe8eadd4539055086b/users/2f123090eba448aca2d02c2111f80579/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:06.698248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f8173-ed2a-43d5-a23e-d7d51b772b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:06.703508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f8173-ed2a-43d5-a23e-d7d51b772b3a service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:06.788896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 882/1782] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:06 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:07.203390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a98279-2b53-4a77-a6b6-803115a18a67 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:07.395173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:07.436856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a98279-2b53-4a77-a6b6-803115a18a67 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:07.456117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9463715-0a0e-4865-8d5b-4179431e304e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:07.459762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463715-0a0e-4865-8d5b-4179431e304e service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:07.462084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463715-0a0e-4865-8d5b-4179431e304e service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:07.464317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463715-0a0e-4865-8d5b-4179431e304e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:07.705271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9463715-0a0e-4865-8d5b-4179431e304e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:07.707257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9463715-0a0e-4865-8d5b-4179431e304e service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:07.721329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 901/1783] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:09.812971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001940e5-918f-461f-8d6d-e831e0c7b1c0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:09.813512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001940e5-918f-461f-8d6d-e831e0c7b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:09.813512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001940e5-918f-461f-8d6d-e831e0c7b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:09.909315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-001940e5-918f-461f-8d6d-e831e0c7b1c0 None None] MFA Rules not processed for user `2f123090eba448aca2d02c2111f80579`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:09.910585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-001940e5-918f-461f-8d6d-e831e0c7b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:10.016851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:10.070128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 883/1784] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:10.082879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9463715-0a0e-4865-8d5b-4179431e304e service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:10.093379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9463715-0a0e-4865-8d5b-4179431e304e service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:10.103967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e987-4732-4962-a983-2229d5f507e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:10.104888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e987-4732-4962-a983-2229d5f507e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:10.105323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e987-4732-4962-a983-2229d5f507e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:10.105735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e987-4732-4962-a983-2229d5f507e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:10.254907 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41e987-4732-4962-a983-2229d5f507e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:10.257014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41e987-4732-4962-a983-2229d5f507e1 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:10.270760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 902/1785] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:14.179474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001940e5-918f-461f-8d6d-e831e0c7b1c0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:14.180912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-001940e5-918f-461f-8d6d-e831e0c7b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:14.201570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001940e5-918f-461f-8d6d-e831e0c7b1c0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:14.206778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21b211-8687-4b61-b8ce-8d89a9b54f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:14.207469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21b211-8687-4b61-b8ce-8d89a9b54f2f service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:14.207914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21b211-8687-4b61-b8ce-8d89a9b54f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:14.208191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21b211-8687-4b61-b8ce-8d89a9b54f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:14.225768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21b211-8687-4b61-b8ce-8d89a9b54f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:14.227326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21b211-8687-4b61-b8ce-8d89a9b54f2f service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:14.265004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 884/1786] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:14.708292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e987-4732-4962-a983-2229d5f507e1 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:14.779265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e987-4732-4962-a983-2229d5f507e1 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:14.784901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:14.785756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:14.786167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:14.786631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] PATH_INFO: `/v3/users/0168d3a277a547f09f08a64c090c3191` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:14.788458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0168d3a277a547f09f08a64c090c3191)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:14.790700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:14.842289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:14.856727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0168d3a277a547f09f08a64c090c3191'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:14.874646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0168d3a277a547f09f08a64c090c3191'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:14.875512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0168d3a277a547f09f08a64c090c3191'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:14.890567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0168d3a277a547f09f08a64c090c3191'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:14.891500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0168d3a277a547f09f08a64c090c3191'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:14.907262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 903/1787] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:14 2022] DELETE /identity/v3/users/0168d3a277a547f09f08a64c090c3191 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:14.916913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224a387-0715-43dd-9486-96eec77c49d1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:14.917318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224a387-0715-43dd-9486-96eec77c49d1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:14.917847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224a387-0715-43dd-9486-96eec77c49d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:14.958892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b224a387-0715-43dd-9486-96eec77c49d1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:14.960157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b224a387-0715-43dd-9486-96eec77c49d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:15.036827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 885/1788] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:15.047009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:15.048460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:15.155853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d2384-99b2-475c-b9e1-8a9b6eab7a00 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:15.160175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137e197-ee4a-415f-97e3-e08f55b529e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:15.160797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137e197-ee4a-415f-97e3-e08f55b529e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:15.161072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137e197-ee4a-415f-97e3-e08f55b529e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:15.161348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137e197-ee4a-415f-97e3-e08f55b529e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:15.172162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137e197-ee4a-415f-97e3-e08f55b529e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:15.173612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137e197-ee4a-415f-97e3-e08f55b529e6 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:15.189725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 904/1789] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:15.988306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224a387-0715-43dd-9486-96eec77c49d1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:15.990011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b224a387-0715-43dd-9486-96eec77c49d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:16.011913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224a387-0715-43dd-9486-96eec77c49d1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:16.021093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:16.022017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:16.022485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:16.023655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] PATH_INFO: `/v3/projects/def0b40d176a4213b5e9c0e7a6778c02` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:16.025812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=def0b40d176a4213b5e9c0e7a6778c02)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:16.028301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:16.116431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def0b40d176a4213b5e9c0e7a6778c02', 'request_id': 'req-ed01a894-7796-4dcc-aad8-d6195db91c6a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.123875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def0b40d176a4213b5e9c0e7a6778c02', 'request_id': 'req-ed01a894-7796-4dcc-aad8-d6195db91c6a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.134000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def0b40d176a4213b5e9c0e7a6778c02', 'request_id': 'req-ed01a894-7796-4dcc-aad8-d6195db91c6a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.153073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] The token cache is being invalidate because project def0b40d176a4213b5e9c0e7a6778c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:16.153465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.154228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.156276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 886/1790] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:15 2022] DELETE /identity/v3/projects/def0b40d176a4213b5e9c0e7a6778c02 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:16.160329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137e197-ee4a-415f-97e3-e08f55b529e6 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:16.185581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137e197-ee4a-415f-97e3-e08f55b529e6 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:16.270632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:16.271447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:16.271729 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:16.272170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] PATH_INFO: `/v3/users/4b6e4fc6c8d444609a5805c4a730c63f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:16.274309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b6e4fc6c8d444609a5805c4a730c63f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:16.276768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:16.329378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b6e4fc6c8d444609a5805c4a730c63f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.337851 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b6e4fc6c8d444609a5805c4a730c63f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.338546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b6e4fc6c8d444609a5805c4a730c63f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.343437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:16.356713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b6e4fc6c8d444609a5805c4a730c63f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.357399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5b34830b-fd32-490a-b25a-1ecbdf473caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b6e4fc6c8d444609a5805c4a730c63f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.371495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 905/1791] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:16 2022] DELETE /identity/v3/users/4b6e4fc6c8d444609a5805c4a730c63f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:16.676623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:16.678106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:16.691691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a894-7796-4dcc-aad8-d6195db91c6a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:16.743017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:16.752772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:16.753360 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:16.753642 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:16.753956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] PATH_INFO: `/v3/projects/b295f1b0df2f4dd2a057502fa1599b91` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:16.755130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b295f1b0df2f4dd2a057502fa1599b91)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:16.757467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:16.804295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b295f1b0df2f4dd2a057502fa1599b91', 'request_id': 'req-308f5f74-eec3-4e56-8c6d-04832ae40585'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.810527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b295f1b0df2f4dd2a057502fa1599b91', 'request_id': 'req-308f5f74-eec3-4e56-8c6d-04832ae40585'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.820109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b295f1b0df2f4dd2a057502fa1599b91', 'request_id': 'req-308f5f74-eec3-4e56-8c6d-04832ae40585'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.830214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] The token cache is being invalidate because project b295f1b0df2f4dd2a057502fa1599b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:16.830493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.831137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-308f5f74-eec3-4e56-8c6d-04832ae40585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:16.832625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 887/1792] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:16 2022] DELETE /identity/v3/projects/b295f1b0df2f4dd2a057502fa1599b91 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:16.837750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26aed6-3d70-4ea8-b09c-eecb66e339b0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:16.838065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26aed6-3d70-4ea8-b09c-eecb66e339b0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:16.838342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26aed6-3d70-4ea8-b09c-eecb66e339b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:16.859930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9f26aed6-3d70-4ea8-b09c-eecb66e339b0 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:16.861076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9f26aed6-3d70-4ea8-b09c-eecb66e339b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:16.924348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 906/1793] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:16.932385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e94446-1478-4990-b9ee-ca622d956fec None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:16.932696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e94446-1478-4990-b9ee-ca622d956fec None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:16.932919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e94446-1478-4990-b9ee-ca622d956fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:16.953763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-88e94446-1478-4990-b9ee-ca622d956fec None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:16.954983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88e94446-1478-4990-b9ee-ca622d956fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:16.963076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 888/1794] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:16.967530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26aed6-3d70-4ea8-b09c-eecb66e339b0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:16.977808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26aed6-3d70-4ea8-b09c-eecb66e339b0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:16.983222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:16.983826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:16.984104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:16.984367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:16.985013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:16.985310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:16.986936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:16.999774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 907/1795] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:35:17.005098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e94446-1478-4990-b9ee-ca622d956fec None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.014564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e94446-1478-4990-b9ee-ca622d956fec None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.020048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c233c-5782-4225-bb75-268fb80ee35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.020682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c233c-5782-4225-bb75-268fb80ee35d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.020953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c233c-5782-4225-bb75-268fb80ee35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.021216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c233c-5782-4225-bb75-268fb80ee35d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.021845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c233c-5782-4225-bb75-268fb80ee35d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.024028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c233c-5782-4225-bb75-268fb80ee35d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.053938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 889/1796] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:17 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:17.057911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.069085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b7060-d442-4745-b5d9-1150d46ba825 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.134139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b405f-9711-4899-abf9-1585fde37e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.134828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b405f-9711-4899-abf9-1585fde37e00 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.135107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b405f-9711-4899-abf9-1585fde37e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.135379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b405f-9711-4899-abf9-1585fde37e00 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.136032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b405f-9711-4899-abf9-1585fde37e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.137976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b405f-9711-4899-abf9-1585fde37e00 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.160636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 908/1797] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:17 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:17.164751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c233c-5782-4225-bb75-268fb80ee35d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.177336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c233c-5782-4225-bb75-268fb80ee35d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.182103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63d409-880a-40d0-a2c5-e8b89e7c8932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.182703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d409-880a-40d0-a2c5-e8b89e7c8932 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.183008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d409-880a-40d0-a2c5-e8b89e7c8932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.183274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d409-880a-40d0-a2c5-e8b89e7c8932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.183918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63d409-880a-40d0-a2c5-e8b89e7c8932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.185553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63d409-880a-40d0-a2c5-e8b89e7c8932 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.196517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 890/1798] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:17.200626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b405f-9711-4899-abf9-1585fde37e00 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.211157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b405f-9711-4899-abf9-1585fde37e00 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.216530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e084e1e-758e-4dbe-b918-e8f69095f813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.217143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e084e1e-758e-4dbe-b918-e8f69095f813 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.217424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e084e1e-758e-4dbe-b918-e8f69095f813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.217672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e084e1e-758e-4dbe-b918-e8f69095f813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.218361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e084e1e-758e-4dbe-b918-e8f69095f813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.220768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e084e1e-758e-4dbe-b918-e8f69095f813 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.232148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 909/1799] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:17.236477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63d409-880a-40d0-a2c5-e8b89e7c8932 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.247301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63d409-880a-40d0-a2c5-e8b89e7c8932 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.252197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0aae8-e3b8-43d6-9da2-6d7eb6e99f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.252866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0aae8-e3b8-43d6-9da2-6d7eb6e99f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.253137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0aae8-e3b8-43d6-9da2-6d7eb6e99f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.253386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0aae8-e3b8-43d6-9da2-6d7eb6e99f7a admin admin] PATH_INFO: `/v3/projects/59117ee540614016acb7e77a1edf4806/users/8bedfb7dbaab435baa9fff06c48f23bf/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.269594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0aae8-e3b8-43d6-9da2-6d7eb6e99f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=59117ee540614016acb7e77a1edf4806, user_id=8bedfb7dbaab435baa9fff06c48f23bf, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.272642 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0aae8-e3b8-43d6-9da2-6d7eb6e99f7a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.287375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 891/1800] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:17 2022] PUT /identity/v3/projects/59117ee540614016acb7e77a1edf4806/users/8bedfb7dbaab435baa9fff06c48f23bf/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:17.297757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6492-9f33-4781-b26f-5567c251fe51 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.298051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6492-9f33-4781-b26f-5567c251fe51 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.298328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6492-9f33-4781-b26f-5567c251fe51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.327299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-241c6492-9f33-4781-b26f-5567c251fe51 None None] MFA Rules not processed for user `8bedfb7dbaab435baa9fff06c48f23bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:17.328516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-241c6492-9f33-4781-b26f-5567c251fe51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:17.360575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:17.398477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 910/1801] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:17.403221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0aae8-e3b8-43d6-9da2-6d7eb6e99f7a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.472337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0aae8-e3b8-43d6-9da2-6d7eb6e99f7a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.477044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575456ee-c008-40a3-b2e2-f6f3364eae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.477660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575456ee-c008-40a3-b2e2-f6f3364eae22 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.477940 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575456ee-c008-40a3-b2e2-f6f3364eae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.478212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575456ee-c008-40a3-b2e2-f6f3364eae22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.478894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575456ee-c008-40a3-b2e2-f6f3364eae22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.480798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575456ee-c008-40a3-b2e2-f6f3364eae22 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.513578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 892/1802] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:17 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:17.515230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c6492-9f33-4781-b26f-5567c251fe51 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.526760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c6492-9f33-4781-b26f-5567c251fe51 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.590558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f3dc5-1d46-45e9-9451-b4b47f86a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.591351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f3dc5-1d46-45e9-9451-b4b47f86a1ed admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.591628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f3dc5-1d46-45e9-9451-b4b47f86a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.591897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f3dc5-1d46-45e9-9451-b4b47f86a1ed admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.592536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f3dc5-1d46-45e9-9451-b4b47f86a1ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.594850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f3dc5-1d46-45e9-9451-b4b47f86a1ed admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.625580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 911/1803] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:17 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:17.629666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575456ee-c008-40a3-b2e2-f6f3364eae22 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.636556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:17.640233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575456ee-c008-40a3-b2e2-f6f3364eae22 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.646324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0232df1-f0bd-4a61-84ab-dae8230764f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.647008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0232df1-f0bd-4a61-84ab-dae8230764f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.647426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0232df1-f0bd-4a61-84ab-dae8230764f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.648054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0232df1-f0bd-4a61-84ab-dae8230764f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.648900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0232df1-f0bd-4a61-84ab-dae8230764f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.650740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0232df1-f0bd-4a61-84ab-dae8230764f3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.661514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 893/1804] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:17.665637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f3dc5-1d46-45e9-9451-b4b47f86a1ed admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.675954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f3dc5-1d46-45e9-9451-b4b47f86a1ed admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.681187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae61fd-24f7-437b-9da3-b4d28b6fd02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.681790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae61fd-24f7-437b-9da3-b4d28b6fd02b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.682059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae61fd-24f7-437b-9da3-b4d28b6fd02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.682326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae61fd-24f7-437b-9da3-b4d28b6fd02b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.683019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae61fd-24f7-437b-9da3-b4d28b6fd02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.684607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae61fd-24f7-437b-9da3-b4d28b6fd02b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.695769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 912/1805] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:17.699971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0232df1-f0bd-4a61-84ab-dae8230764f3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.711656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0232df1-f0bd-4a61-84ab-dae8230764f3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.716516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667cd95-21ff-40f4-80a6-3b12936ddb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.717161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667cd95-21ff-40f4-80a6-3b12936ddb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.717437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667cd95-21ff-40f4-80a6-3b12936ddb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.717731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667cd95-21ff-40f4-80a6-3b12936ddb88 admin admin] PATH_INFO: `/v3/projects/1ddebfb00af847c2a35d85dc7e65f88f/users/0e317026efe94f84931f922278b61e46/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.733664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667cd95-21ff-40f4-80a6-3b12936ddb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddebfb00af847c2a35d85dc7e65f88f, user_id=0e317026efe94f84931f922278b61e46, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.736385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667cd95-21ff-40f4-80a6-3b12936ddb88 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:17.756103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 894/1806] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:17 2022] PUT /identity/v3/projects/1ddebfb00af847c2a35d85dc7e65f88f/users/0e317026efe94f84931f922278b61e46/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:17.773648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4837dea-27c0-4196-9ebc-045d603b4431 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.773812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4837dea-27c0-4196-9ebc-045d603b4431 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.774113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4837dea-27c0-4196-9ebc-045d603b4431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.803625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a4837dea-27c0-4196-9ebc-045d603b4431 None None] MFA Rules not processed for user `0e317026efe94f84931f922278b61e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:17.804828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a4837dea-27c0-4196-9ebc-045d603b4431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:17.880829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 913/1807] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:17.886202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667cd95-21ff-40f4-80a6-3b12936ddb88 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:17.958454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667cd95-21ff-40f4-80a6-3b12936ddb88 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:17.963984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abd2e6-a729-42f4-b089-91e53401ab39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:17.964609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abd2e6-a729-42f4-b089-91e53401ab39 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:17.964889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abd2e6-a729-42f4-b089-91e53401ab39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:17.965162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1abd2e6-a729-42f4-b089-91e53401ab39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:17.965801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abd2e6-a729-42f4-b089-91e53401ab39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:17.967754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1abd2e6-a729-42f4-b089-91e53401ab39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:18.004729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 895/1808] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:17 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:18.006179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4837dea-27c0-4196-9ebc-045d603b4431 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:18.017484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4837dea-27c0-4196-9ebc-045d603b4431 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:18.058911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:18.081026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def68eb0-5478-4309-b08b-44ad0936ddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:18.081650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def68eb0-5478-4309-b08b-44ad0936ddfb admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:18.081928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def68eb0-5478-4309-b08b-44ad0936ddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:18.082198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def68eb0-5478-4309-b08b-44ad0936ddfb admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:18.082883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def68eb0-5478-4309-b08b-44ad0936ddfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:18.084941 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def68eb0-5478-4309-b08b-44ad0936ddfb admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:18.113010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 914/1809] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:18 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:18.116061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abd2e6-a729-42f4-b089-91e53401ab39 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:18.126497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1abd2e6-a729-42f4-b089-91e53401ab39 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:18.131041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebd73-1bdb-4228-822a-874ee51768c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:18.131894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebd73-1bdb-4228-822a-874ee51768c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:18.132334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebd73-1bdb-4228-822a-874ee51768c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:18.132608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebd73-1bdb-4228-822a-874ee51768c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:18.133257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaebd73-1bdb-4228-822a-874ee51768c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:18.134887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaebd73-1bdb-4228-822a-874ee51768c7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:18.145438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 896/1810] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:18.149880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def68eb0-5478-4309-b08b-44ad0936ddfb admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:18.160573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def68eb0-5478-4309-b08b-44ad0936ddfb admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:18.166467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0605118-eba2-45d3-9569-a02354482f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:18.167188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0605118-eba2-45d3-9569-a02354482f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:18.167465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0605118-eba2-45d3-9569-a02354482f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:18.167722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0605118-eba2-45d3-9569-a02354482f43 admin admin] PATH_INFO: `/v3/projects/25515cb671624181823ab0158067a332/users/624c1b0552e44feeb883b26998e8b221/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:18.184365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0605118-eba2-45d3-9569-a02354482f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25515cb671624181823ab0158067a332, user_id=624c1b0552e44feeb883b26998e8b221, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:18.186973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0605118-eba2-45d3-9569-a02354482f43 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:18.201403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 915/1811] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:18 2022] PUT /identity/v3/projects/25515cb671624181823ab0158067a332/users/624c1b0552e44feeb883b26998e8b221/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:18.205520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebd73-1bdb-4228-822a-874ee51768c7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:18.216072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebd73-1bdb-4228-822a-874ee51768c7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:18.276480 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e508e6f-780d-4df0-a97a-0fdb43e8de0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:18.277130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508e6f-780d-4df0-a97a-0fdb43e8de0c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:18.277380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508e6f-780d-4df0-a97a-0fdb43e8de0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:18.277652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508e6f-780d-4df0-a97a-0fdb43e8de0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:18.278307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e508e6f-780d-4df0-a97a-0fdb43e8de0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:18.279940 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e508e6f-780d-4df0-a97a-0fdb43e8de0c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:18.291658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 897/1812] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:18.296454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0605118-eba2-45d3-9569-a02354482f43 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:18.307000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0605118-eba2-45d3-9569-a02354482f43 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:18.311741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:18.312410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:18.312823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:18.313099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] PATH_INFO: `/v3/projects/25515cb671624181823ab0158067a332/users/624c1b0552e44feeb883b26998e8b221/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:18.315023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] RBAC: Authorizing `identity:create_grant(project_id=25515cb671624181823ab0158067a332, user_id=624c1b0552e44feeb883b26998e8b221, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:18.317644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:18.328746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 916/1813] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:18 2022] PUT /identity/v3/projects/25515cb671624181823ab0158067a332/users/624c1b0552e44feeb883b26998e8b221/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:18.338491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5062-6033-45b7-b62f-eec3ca3ea7c5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:18.338824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5062-6033-45b7-b62f-eec3ca3ea7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:18.339099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5062-6033-45b7-b62f-eec3ca3ea7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:18.367432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-34ab5062-6033-45b7-b62f-eec3ca3ea7c5 None None] MFA Rules not processed for user `624c1b0552e44feeb883b26998e8b221`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:18.368739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34ab5062-6033-45b7-b62f-eec3ca3ea7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:18.397728 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:18.438678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 898/1814] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:18.450116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:18.452047 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:18.518045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032432f-e453-416c-80a1-eabece41861d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:18.524023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d42b9-7807-470f-b599-3444d2f46363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:18.524628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d42b9-7807-470f-b599-3444d2f46363 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:18.524866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d42b9-7807-470f-b599-3444d2f46363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:18.525150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d42b9-7807-470f-b599-3444d2f46363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:18.535030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d42b9-7807-470f-b599-3444d2f46363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:18.536389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d42b9-7807-470f-b599-3444d2f46363 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:18.546067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 917/1815] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:18.594239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab5062-6033-45b7-b62f-eec3ca3ea7c5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:18.596077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34ab5062-6033-45b7-b62f-eec3ca3ea7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:18.609647 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab5062-6033-45b7-b62f-eec3ca3ea7c5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:18.614799 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae34b8-b45f-4c96-a8be-503bf121b4cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:18.615545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae34b8-b45f-4c96-a8be-503bf121b4cf service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:18.616235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae34b8-b45f-4c96-a8be-503bf121b4cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:18.617306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae34b8-b45f-4c96-a8be-503bf121b4cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:18.680405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae34b8-b45f-4c96-a8be-503bf121b4cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:18.682167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae34b8-b45f-4c96-a8be-503bf121b4cf service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:18.698970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 899/1816] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:20.008117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d74e1-6bda-4bd0-a8e7-e0197881b0b9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:20.008691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d74e1-6bda-4bd0-a8e7-e0197881b0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:20.008691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d74e1-6bda-4bd0-a8e7-e0197881b0b9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:20.010016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 918/1817] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:35:20.019713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c017-1551-4ced-809e-c8390e6c6ddc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:20.020156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c017-1551-4ced-809e-c8390e6c6ddc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:20.020260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c017-1551-4ced-809e-c8390e6c6ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:20.043614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5201c017-1551-4ced-809e-c8390e6c6ddc None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:20.044824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5201c017-1551-4ced-809e-c8390e6c6ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:20.116590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 900/1818] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:35:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:20.128346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d74e1-6bda-4bd0-a8e7-e0197881b0b9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:20.130238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f09d74e1-6bda-4bd0-a8e7-e0197881b0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:20.148190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d74e1-6bda-4bd0-a8e7-e0197881b0b9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:20.153655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469e6ae-28ee-4d90-975b-abdfa92fa8e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:20.154274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469e6ae-28ee-4d90-975b-abdfa92fa8e6 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:20.154546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469e6ae-28ee-4d90-975b-abdfa92fa8e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:20.154844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469e6ae-28ee-4d90-975b-abdfa92fa8e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:20.164868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469e6ae-28ee-4d90-975b-abdfa92fa8e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:20.166517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469e6ae-28ee-4d90-975b-abdfa92fa8e6 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:20.177435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 919/1819] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:23.697630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c017-1551-4ced-809e-c8390e6c6ddc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:23.699071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5201c017-1551-4ced-809e-c8390e6c6ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:23.833128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:23.848803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c017-1551-4ced-809e-c8390e6c6ddc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:23.856380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:23.857359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:23.857853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:23.858272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] PATH_INFO: `/v3/users/eaf82e5e73d04955962c300d055376e6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:23.860933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf82e5e73d04955962c300d055376e6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:23.863904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:23.963303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf82e5e73d04955962c300d055376e6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:23.974895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf82e5e73d04955962c300d055376e6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:23.975715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf82e5e73d04955962c300d055376e6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:23.994123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf82e5e73d04955962c300d055376e6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:23.994942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf82e5e73d04955962c300d055376e6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:24.023976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 901/1820] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:23 2022] DELETE /identity/v3/users/eaf82e5e73d04955962c300d055376e6 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:24.026029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f3d39-3566-497a-b4bc-3e51de5ff6cf None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:24.026497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f3d39-3566-497a-b4bc-3e51de5ff6cf None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:24.026614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f3d39-3566-497a-b4bc-3e51de5ff6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:24.062459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f90f3d39-3566-497a-b4bc-3e51de5ff6cf None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:24.064891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f90f3d39-3566-497a-b4bc-3e51de5ff6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:24.150054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 920/1821] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:24.160507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:24.162542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:24.248152 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869a96f-c8af-4110-8b56-746b0fca1cf2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:24.260275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:24.261048 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:24.261322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:24.261665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:24.277053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:24.279313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:24.297577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 902/1822] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:35:25.347167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f3d39-3566-497a-b4bc-3e51de5ff6cf None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:25.348973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f90f3d39-3566-497a-b4bc-3e51de5ff6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:25.363431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f3d39-3566-497a-b4bc-3e51de5ff6cf None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:25.370342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:25.372338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:25.372929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:25.373394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] PATH_INFO: `/v3/projects/25a70b14a07b44a68f0a998219eb5587` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:25.385280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a70b14a07b44a68f0a998219eb5587)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:25.391512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:25.435838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:25.438236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:25.450428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:25.459657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978785e4-2990-48b6-b089-55b36c3be4a3 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:25.478069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:25.481104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:25.481717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:25.481989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] PATH_INFO: `/v3/users/8bedfb7dbaab435baa9fff06c48f23bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:25.485266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bedfb7dbaab435baa9fff06c48f23bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:25.491523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:25.513962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a70b14a07b44a68f0a998219eb5587', 'request_id': 'req-d1073877-b371-4af7-b3b0-bbfbb0160e63'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.530125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a70b14a07b44a68f0a998219eb5587', 'request_id': 'req-d1073877-b371-4af7-b3b0-bbfbb0160e63'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.543622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a70b14a07b44a68f0a998219eb5587', 'request_id': 'req-d1073877-b371-4af7-b3b0-bbfbb0160e63'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.575830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] The token cache is being invalidate because project 25a70b14a07b44a68f0a998219eb5587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:25.576184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.577847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.580005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bedfb7dbaab435baa9fff06c48f23bf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.580837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 921/1823] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:25 2022] DELETE /identity/v3/projects/25a70b14a07b44a68f0a998219eb5587 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:25.585314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:25.588561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:25.597071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bedfb7dbaab435baa9fff06c48f23bf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.598707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bedfb7dbaab435baa9fff06c48f23bf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.616979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bedfb7dbaab435baa9fff06c48f23bf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.618080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd07b7df-9b58-42b0-a359-5a294879e7be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bedfb7dbaab435baa9fff06c48f23bf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.619643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:25.640932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 903/1824] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:25 2022] DELETE /identity/v3/users/8bedfb7dbaab435baa9fff06c48f23bf => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:25.645443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75310cd-1d54-4d63-acf1-a53b4e0e3b22 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:25.645723 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75310cd-1d54-4d63-acf1-a53b4e0e3b22 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:25.646002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75310cd-1d54-4d63-acf1-a53b4e0e3b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:25.670127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d75310cd-1d54-4d63-acf1-a53b4e0e3b22 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:25.671531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d75310cd-1d54-4d63-acf1-a53b4e0e3b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:25.722627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1073877-b371-4af7-b3b0-bbfbb0160e63 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:25.777243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 904/1825] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:25.786523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75310cd-1d54-4d63-acf1-a53b4e0e3b22 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:25.788200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d75310cd-1d54-4d63-acf1-a53b4e0e3b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:25.799947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:25.799947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:25.799947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:25.799947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] PATH_INFO: `/v3/users/d53c8f67066047cf957eefb5e49bdf9b` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:25.802426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53c8f67066047cf957eefb5e49bdf9b)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:25.804424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:25.860343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53c8f67066047cf957eefb5e49bdf9b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.868110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75310cd-1d54-4d63-acf1-a53b4e0e3b22 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:25.871191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53c8f67066047cf957eefb5e49bdf9b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.872184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:25.872320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53c8f67066047cf957eefb5e49bdf9b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.872945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:25.873269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:25.873515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:25.873614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:25.875451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:25.890897 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:25.892346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:25.896503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53c8f67066047cf957eefb5e49bdf9b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.897268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-00615782-e57e-4454-8407-36aa548d3dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53c8f67066047cf957eefb5e49bdf9b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:25.906933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 905/1826] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:35:25.919535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 922/1827] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:25 2022] DELETE /identity/v3/users/d53c8f67066047cf957eefb5e49bdf9b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:26.465195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:26.468218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:26.491833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5c8c2-76d8-4540-aa83-8f947e349761 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:26.703751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:26.704299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:26.704573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:26.704851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] PATH_INFO: `/v3/projects/ade5fe83294847c78981650d0113d10c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:26.709694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ade5fe83294847c78981650d0113d10c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:26.711848 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:26.811338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:26.844124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ade5fe83294847c78981650d0113d10c', 'request_id': 'req-141d19a5-9e87-482c-b667-d0d07ea82ea8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:26.851847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ade5fe83294847c78981650d0113d10c', 'request_id': 'req-141d19a5-9e87-482c-b667-d0d07ea82ea8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:26.872659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ade5fe83294847c78981650d0113d10c', 'request_id': 'req-141d19a5-9e87-482c-b667-d0d07ea82ea8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:26.900146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] The token cache is being invalidate because project ade5fe83294847c78981650d0113d10c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:26.900953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:26.901783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:26.904952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 906/1828] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:26 2022] DELETE /identity/v3/projects/ade5fe83294847c78981650d0113d10c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:26.910288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f51b-cb7e-48df-b683-c54df3d579e9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:26.910793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f51b-cb7e-48df-b683-c54df3d579e9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:26.911002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f51b-cb7e-48df-b683-c54df3d579e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:26.941942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-38a7f51b-cb7e-48df-b683-c54df3d579e9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:26.943530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-38a7f51b-cb7e-48df-b683-c54df3d579e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:26.957985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:26.959723 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:27.075794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d19a5-9e87-482c-b667-d0d07ea82ea8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.077064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 923/1829] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:27.082457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.083269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.083698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.084117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] PATH_INFO: `/v3/projects/59117ee540614016acb7e77a1edf4806` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.084508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707bf8b-e422-491f-9472-650b80293fe9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.084791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707bf8b-e422-491f-9472-650b80293fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.085059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707bf8b-e422-491f-9472-650b80293fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.085869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59117ee540614016acb7e77a1edf4806)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.088270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.108261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8707bf8b-e422-491f-9472-650b80293fe9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:27.109473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8707bf8b-e422-491f-9472-650b80293fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:27.118752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 924/1830] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:27.123114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707bf8b-e422-491f-9472-650b80293fe9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.134313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707bf8b-e422-491f-9472-650b80293fe9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.144699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59117ee540614016acb7e77a1edf4806', 'request_id': 'req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.155555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59117ee540614016acb7e77a1edf4806', 'request_id': 'req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.165205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59117ee540614016acb7e77a1edf4806', 'request_id': 'req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.178039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] The token cache is being invalidate because project 59117ee540614016acb7e77a1edf4806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:27.178391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.179365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.181556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 907/1831] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:26 2022] DELETE /identity/v3/projects/59117ee540614016acb7e77a1edf4806 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:27.185456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.187083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:27.196294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.196976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.197252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.197677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.198468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:27.198818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.200427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.215218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 925/1832] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:35:27.220492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.222362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:27.235676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093c27-daae-4815-8fef-f5682662e5c6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.242418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711486d-9503-451c-ac73-cb6b974f0473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.243463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711486d-9503-451c-ac73-cb6b974f0473 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.243886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711486d-9503-451c-ac73-cb6b974f0473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.244332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711486d-9503-451c-ac73-cb6b974f0473 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.245149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711486d-9503-451c-ac73-cb6b974f0473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.247375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711486d-9503-451c-ac73-cb6b974f0473 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.262477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:27.276532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c18fe-5b4e-4b6b-99e2-c1ab48bfbfe5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.281297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 926/1833] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:27 2022] POST /identity/v3/projects => generated 401 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:27.285428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711486d-9503-451c-ac73-cb6b974f0473 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.286015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.287378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.287853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.288808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] PATH_INFO: `/v3/users/0e317026efe94f84931f922278b61e46` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.290218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e317026efe94f84931f922278b61e46)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.293364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.333006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:27.343968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e317026efe94f84931f922278b61e46'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.352864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e317026efe94f84931f922278b61e46'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.353536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e317026efe94f84931f922278b61e46'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.357632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711486d-9503-451c-ac73-cb6b974f0473 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.362960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244c522-919e-455e-858a-7f545a96c673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.363649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c522-919e-455e-858a-7f545a96c673 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.363928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c522-919e-455e-858a-7f545a96c673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.364197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c522-919e-455e-858a-7f545a96c673 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.364832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244c522-919e-455e-858a-7f545a96c673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.366905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244c522-919e-455e-858a-7f545a96c673 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.367194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e317026efe94f84931f922278b61e46'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.367941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e317026efe94f84931f922278b61e46'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:27.385026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 908/1834] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:27 2022] DELETE /identity/v3/users/0e317026efe94f84931f922278b61e46 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:27.402273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 927/1835] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:27 2022] POST /identity/v3/users => generated 301 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:27.403493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.407844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:27.514648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b7337-02e1-4b26-a213-20174e0a11a8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.529337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af4eb6-a183-44f2-8167-47eb2c10800b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.529951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af4eb6-a183-44f2-8167-47eb2c10800b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.530236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af4eb6-a183-44f2-8167-47eb2c10800b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.530525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af4eb6-a183-44f2-8167-47eb2c10800b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.532141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af4eb6-a183-44f2-8167-47eb2c10800b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.533104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af4eb6-a183-44f2-8167-47eb2c10800b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.550464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 909/1836] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:27.559131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244c522-919e-455e-858a-7f545a96c673 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.572881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244c522-919e-455e-858a-7f545a96c673 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.579453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ff136-e545-49c4-bdcc-4c5b9a8b40fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.580015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff136-e545-49c4-bdcc-4c5b9a8b40fe admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.580294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff136-e545-49c4-bdcc-4c5b9a8b40fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.580570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ff136-e545-49c4-bdcc-4c5b9a8b40fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.581244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ff136-e545-49c4-bdcc-4c5b9a8b40fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.583324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ff136-e545-49c4-bdcc-4c5b9a8b40fe admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.600727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 928/1837] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:27.609227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af4eb6-a183-44f2-8167-47eb2c10800b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.620152 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af4eb6-a183-44f2-8167-47eb2c10800b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.625401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.626320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.629813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.630303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] PATH_INFO: `/v3/projects/0b68ad5fb6c546619bb5b18f6358de78/users/bc2a4e1ca0784a4d954f6805a9c4c47b/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.647238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b68ad5fb6c546619bb5b18f6358de78, user_id=bc2a4e1ca0784a4d954f6805a9c4c47b, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.650328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.672273 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 910/1838] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:27 2022] PUT /identity/v3/projects/0b68ad5fb6c546619bb5b18f6358de78/users/bc2a4e1ca0784a4d954f6805a9c4c47b/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:27.684225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c238-0323-42d9-9841-35da327e7a85 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.686019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c238-0323-42d9-9841-35da327e7a85 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.686019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74c238-0323-42d9-9841-35da327e7a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.744535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0e74c238-0323-42d9-9841-35da327e7a85 None None] MFA Rules not processed for user `bc2a4e1ca0784a4d954f6805a9c4c47b`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:27.746778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0e74c238-0323-42d9-9841-35da327e7a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:27.864360 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.866109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:27.868184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 929/1839] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:27 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:27.872598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74c238-0323-42d9-9841-35da327e7a85 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:27.955637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74c238-0323-42d9-9841-35da327e7a85 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:27.961009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36aa3-5701-4673-8fd6-7e544ce79947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:27.962297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36aa3-5701-4673-8fd6-7e544ce79947 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:27.962711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36aa3-5701-4673-8fd6-7e544ce79947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:27.963069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36aa3-5701-4673-8fd6-7e544ce79947 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:27.963883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36aa3-5701-4673-8fd6-7e544ce79947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:27.974567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36aa3-5701-4673-8fd6-7e544ce79947 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:27.976424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:28.026828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 930/1840] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:27 2022] POST /identity/v3/projects => generated 401 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:28.029614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2af66-60db-41dd-87c9-ea7dd1466129 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:28.031212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36aa3-5701-4673-8fd6-7e544ce79947 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:28.077013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:28.119765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36aa3-5701-4673-8fd6-7e544ce79947 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:28.125413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f6e72-7325-46f9-94a8-9797a55c1b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:28.126282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f6e72-7325-46f9-94a8-9797a55c1b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:28.126787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f6e72-7325-46f9-94a8-9797a55c1b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:28.127246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f6e72-7325-46f9-94a8-9797a55c1b11 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:28.128145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f6e72-7325-46f9-94a8-9797a55c1b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:28.131179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f6e72-7325-46f9-94a8-9797a55c1b11 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:28.144104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:28.145090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:28.145209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:28.145412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] PATH_INFO: `/v3/projects/1ddebfb00af847c2a35d85dc7e65f88f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:28.146890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ddebfb00af847c2a35d85dc7e65f88f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:28.152136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:28.164269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 931/1841] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:28 2022] POST /identity/v3/users => generated 307 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:28.168891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f6e72-7325-46f9-94a8-9797a55c1b11 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:28.181536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f6e72-7325-46f9-94a8-9797a55c1b11 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:28.191230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9426d5-ba83-4fe8-8baa-16923f93d580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:28.193114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9426d5-ba83-4fe8-8baa-16923f93d580 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:28.193553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9426d5-ba83-4fe8-8baa-16923f93d580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:28.194091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9426d5-ba83-4fe8-8baa-16923f93d580 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:28.195086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9426d5-ba83-4fe8-8baa-16923f93d580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:28.197071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9426d5-ba83-4fe8-8baa-16923f93d580 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:28.211480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 932/1842] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:28.215975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9426d5-ba83-4fe8-8baa-16923f93d580 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:28.231252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9426d5-ba83-4fe8-8baa-16923f93d580 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:28.244229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ddebfb00af847c2a35d85dc7e65f88f', 'request_id': 'req-4eff356b-711a-4f19-a316-86dc15dee868'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.255847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ddebfb00af847c2a35d85dc7e65f88f', 'request_id': 'req-4eff356b-711a-4f19-a316-86dc15dee868'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.272668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ddebfb00af847c2a35d85dc7e65f88f', 'request_id': 'req-4eff356b-711a-4f19-a316-86dc15dee868'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.307516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] The token cache is being invalidate because project 1ddebfb00af847c2a35d85dc7e65f88f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:28.308784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.310297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.316085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 911/1843] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:27 2022] DELETE /identity/v3/projects/1ddebfb00af847c2a35d85dc7e65f88f => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:28.321243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:28.325155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:28.380784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:28.381555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:28.381749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:28.382022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] PATH_INFO: `/v3/projects/1de531046e80469db3fb9c3c7c494240/users/3be9983e926743bcacb402b291c89d74/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:28.416345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de531046e80469db3fb9c3c7c494240, user_id=3be9983e926743bcacb402b291c89d74, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:28.452292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:28.490813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff356b-711a-4f19-a316-86dc15dee868 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:28.505286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:28.506477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:28.506477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:28.506663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] PATH_INFO: `/v3/users/624c1b0552e44feeb883b26998e8b221` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:28.515889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624c1b0552e44feeb883b26998e8b221)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:28.518045 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:28.526813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 933/1844] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:28 2022] PUT /identity/v3/projects/1de531046e80469db3fb9c3c7c494240/users/3be9983e926743bcacb402b291c89d74/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:28.534703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:28.539131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:28.578790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:28.639986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624c1b0552e44feeb883b26998e8b221'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.660838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624c1b0552e44feeb883b26998e8b221'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.661780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624c1b0552e44feeb883b26998e8b221'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.679577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94b295-8a8f-43c6-aa15-3ca7a0ceb9be admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:28.685747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624c1b0552e44feeb883b26998e8b221'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.686519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624c1b0552e44feeb883b26998e8b221'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:28.706968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 912/1845] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:28 2022] DELETE /identity/v3/users/624c1b0552e44feeb883b26998e8b221 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:28.708912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280b2a3-3558-41ce-be09-33ab5742d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:28.711707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280b2a3-3558-41ce-be09-33ab5742d1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:28.712973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280b2a3-3558-41ce-be09-33ab5742d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:28.713522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280b2a3-3558-41ce-be09-33ab5742d1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:28.714575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280b2a3-3558-41ce-be09-33ab5742d1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:28.717026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280b2a3-3558-41ce-be09-33ab5742d1f3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:28.733584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 934/1846] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:35:28.737540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:28.739043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:28.818400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02555b5e-ba75-40b5-998a-e451531a6bc6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:28.824633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:28.825068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:28.825342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:28.825612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] PATH_INFO: `/v3/projects/1de531046e80469db3fb9c3c7c494240/users/3be9983e926743bcacb402b291c89d74/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:28.827898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de531046e80469db3fb9c3c7c494240, user_id=3be9983e926743bcacb402b291c89d74, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:28.830537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:28.842824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 913/1847] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:28 2022] PUT /identity/v3/projects/1de531046e80469db3fb9c3c7c494240/users/3be9983e926743bcacb402b291c89d74/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:28.855944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cbc0a-da35-48ff-966d-770b17c2a002 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:28.856219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cbc0a-da35-48ff-966d-770b17c2a002 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:28.857029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cbc0a-da35-48ff-966d-770b17c2a002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:28.891698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f54cbc0a-da35-48ff-966d-770b17c2a002 None None] MFA Rules not processed for user `3be9983e926743bcacb402b291c89d74`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:28.892923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f54cbc0a-da35-48ff-966d-770b17c2a002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:28.912747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:28.996036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 935/1848] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:28 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:29.014396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:29.016806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:29.106992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14c3dc-43af-4686-814f-deb097cf3089 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:29.110640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:29.111299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:29.111574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:29.111839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:29.122214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:29.124590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:29.142181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 914/1849] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:29 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:29.244972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cbc0a-da35-48ff-966d-770b17c2a002 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:29.254643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cbc0a-da35-48ff-966d-770b17c2a002 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:29.259216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1755e1e-2871-4003-be73-79325f5b1eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:29.259837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1755e1e-2871-4003-be73-79325f5b1eeb service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:29.260143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1755e1e-2871-4003-be73-79325f5b1eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:29.260411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1755e1e-2871-4003-be73-79325f5b1eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:29.262181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d1755e1e-2871-4003-be73-79325f5b1eeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:29.324637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1755e1e-2871-4003-be73-79325f5b1eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:29.326115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1755e1e-2871-4003-be73-79325f5b1eeb service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:29.336697 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 936/1850] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:29 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:29.394139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:29.395636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:29.443411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:29.473440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc66b5-9fd1-4165-8374-fc64706ecc15 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:29.478476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:29.479143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:29.479421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:29.479690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] PATH_INFO: `/v3/projects/25515cb671624181823ab0158067a332` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:29.480842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25515cb671624181823ab0158067a332)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:29.483514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:29.534056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25515cb671624181823ab0158067a332', 'request_id': 'req-4777226e-82eb-49e2-95e3-b4d00e738df7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:29.542565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25515cb671624181823ab0158067a332', 'request_id': 'req-4777226e-82eb-49e2-95e3-b4d00e738df7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:29.552823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25515cb671624181823ab0158067a332', 'request_id': 'req-4777226e-82eb-49e2-95e3-b4d00e738df7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:29.566170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] The token cache is being invalidate because project 25515cb671624181823ab0158067a332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:29.566862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:29.567693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4777226e-82eb-49e2-95e3-b4d00e738df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:29.574035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 915/1851] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:29 2022] DELETE /identity/v3/projects/25515cb671624181823ab0158067a332 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:29.576338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0f74d-2f41-475d-ab99-57ac701c19fa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:29.576718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0f74d-2f41-475d-ab99-57ac701c19fa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:29.577115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0f74d-2f41-475d-ab99-57ac701c19fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:29.610481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-21a0f74d-2f41-475d-ab99-57ac701c19fa None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:29.611826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-21a0f74d-2f41-475d-ab99-57ac701c19fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:29.728535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 937/1852] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:29.737160 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebdcb9-273c-4e02-9745-c58b2992695c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:29.737529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebdcb9-273c-4e02-9745-c58b2992695c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:29.737869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebdcb9-273c-4e02-9745-c58b2992695c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:29.767214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cdebdcb9-273c-4e02-9745-c58b2992695c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:29.768644 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cdebdcb9-273c-4e02-9745-c58b2992695c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:29.779237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 916/1853] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:29.783615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0f74d-2f41-475d-ab99-57ac701c19fa None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:29.793530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0f74d-2f41-475d-ab99-57ac701c19fa None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:29.798374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:29.799015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:29.799291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:29.799560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:29.800333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:29.800725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:29.802230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:29.815203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 938/1854] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:35:29.827603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebdcb9-273c-4e02-9745-c58b2992695c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:29.841937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebdcb9-273c-4e02-9745-c58b2992695c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:29.848337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639a9c-b330-484b-a254-1c46e7099ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:29.848959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639a9c-b330-484b-a254-1c46e7099ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:29.849236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639a9c-b330-484b-a254-1c46e7099ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:29.849506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639a9c-b330-484b-a254-1c46e7099ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:29.850336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd639a9c-b330-484b-a254-1c46e7099ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:29.853149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd639a9c-b330-484b-a254-1c46e7099ad7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:29.885218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 917/1855] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:29 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:29.891120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:29.907860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beda854-a3d2-4458-a77b-226caf23c886 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:29.954291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:29.974631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9aa385-a490-48fa-a0a4-c0d6b0b4eb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:29.975281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa385-a490-48fa-a0a4-c0d6b0b4eb23 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:29.975565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa385-a490-48fa-a0a4-c0d6b0b4eb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:29.975818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa385-a490-48fa-a0a4-c0d6b0b4eb23 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:29.976742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9aa385-a490-48fa-a0a4-c0d6b0b4eb23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:29.978756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9aa385-a490-48fa-a0a4-c0d6b0b4eb23 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.001517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 939/1856] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:29 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:30.005895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639a9c-b330-484b-a254-1c46e7099ad7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.017735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639a9c-b330-484b-a254-1c46e7099ad7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.022849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93dab1-147f-4167-b408-948693e768ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.023512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93dab1-147f-4167-b408-948693e768ea admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.023883 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93dab1-147f-4167-b408-948693e768ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.024192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93dab1-147f-4167-b408-948693e768ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.024876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93dab1-147f-4167-b408-948693e768ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.026697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93dab1-147f-4167-b408-948693e768ea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.036954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 918/1857] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:30.041344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9aa385-a490-48fa-a0a4-c0d6b0b4eb23 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.051859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9aa385-a490-48fa-a0a4-c0d6b0b4eb23 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.056726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4003595-bc1f-49b3-9a78-91597222e365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.057409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4003595-bc1f-49b3-9a78-91597222e365 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.057591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4003595-bc1f-49b3-9a78-91597222e365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.057858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4003595-bc1f-49b3-9a78-91597222e365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.058503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4003595-bc1f-49b3-9a78-91597222e365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.062853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4003595-bc1f-49b3-9a78-91597222e365 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.078265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 940/1858] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:30.082764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93dab1-147f-4167-b408-948693e768ea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.093510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93dab1-147f-4167-b408-948693e768ea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.099234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef73afd-154e-4707-85d0-00b4ce124236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.099898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73afd-154e-4707-85d0-00b4ce124236 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.100176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73afd-154e-4707-85d0-00b4ce124236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.100445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73afd-154e-4707-85d0-00b4ce124236 admin admin] PATH_INFO: `/v3/projects/0b2788168e1b444fb9d8ab7d3f986371/users/81ff6bd2f3da46378e5cb1ff956e70d6/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.116054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef73afd-154e-4707-85d0-00b4ce124236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2788168e1b444fb9d8ab7d3f986371, user_id=81ff6bd2f3da46378e5cb1ff956e70d6, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.118842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef73afd-154e-4707-85d0-00b4ce124236 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.134622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 919/1859] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:30 2022] PUT /identity/v3/projects/0b2788168e1b444fb9d8ab7d3f986371/users/81ff6bd2f3da46378e5cb1ff956e70d6/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:30.145972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402d5e1-bdc8-45ba-8d17-a024168b9f46 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.146329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402d5e1-bdc8-45ba-8d17-a024168b9f46 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.146604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402d5e1-bdc8-45ba-8d17-a024168b9f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.177706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4402d5e1-bdc8-45ba-8d17-a024168b9f46 None None] MFA Rules not processed for user `81ff6bd2f3da46378e5cb1ff956e70d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:30.180443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4402d5e1-bdc8-45ba-8d17-a024168b9f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:30.262042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 941/1860] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:30.263822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef73afd-154e-4707-85d0-00b4ce124236 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.362819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef73afd-154e-4707-85d0-00b4ce124236 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.369704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b85c4a-b157-423a-a76e-1396186b3564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.370453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b85c4a-b157-423a-a76e-1396186b3564 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.370766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b85c4a-b157-423a-a76e-1396186b3564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.371041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b85c4a-b157-423a-a76e-1396186b3564 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.371683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b85c4a-b157-423a-a76e-1396186b3564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.373807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b85c4a-b157-423a-a76e-1396186b3564 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.387794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:30.420673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 920/1861] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:30 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:30.421810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402d5e1-bdc8-45ba-8d17-a024168b9f46 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.434029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402d5e1-bdc8-45ba-8d17-a024168b9f46 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.512669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7b686-eda1-4b7c-a21d-d217b1ac0686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.513299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b686-eda1-4b7c-a21d-d217b1ac0686 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.513579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b686-eda1-4b7c-a21d-d217b1ac0686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.513866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7b686-eda1-4b7c-a21d-d217b1ac0686 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.514494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7b686-eda1-4b7c-a21d-d217b1ac0686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.516473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7b686-eda1-4b7c-a21d-d217b1ac0686 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.543339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 942/1862] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:30 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:30.547452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b85c4a-b157-423a-a76e-1396186b3564 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.558749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b85c4a-b157-423a-a76e-1396186b3564 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.564371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cad8d-bc4a-4afb-a095-2735738ecc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.565267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cad8d-bc4a-4afb-a095-2735738ecc3b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.565661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cad8d-bc4a-4afb-a095-2735738ecc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.566110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cad8d-bc4a-4afb-a095-2735738ecc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.567116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cad8d-bc4a-4afb-a095-2735738ecc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.569416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cad8d-bc4a-4afb-a095-2735738ecc3b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.582832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 921/1863] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:30.587593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7b686-eda1-4b7c-a21d-d217b1ac0686 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.598430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7b686-eda1-4b7c-a21d-d217b1ac0686 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.603572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b982a-62a0-4ed7-91c6-5d1f3d2e0f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.604182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b982a-62a0-4ed7-91c6-5d1f3d2e0f73 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.604458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b982a-62a0-4ed7-91c6-5d1f3d2e0f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.604724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b982a-62a0-4ed7-91c6-5d1f3d2e0f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.605367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b982a-62a0-4ed7-91c6-5d1f3d2e0f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.607075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b982a-62a0-4ed7-91c6-5d1f3d2e0f73 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.617410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 943/1864] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:30.621652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cad8d-bc4a-4afb-a095-2735738ecc3b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.631726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cad8d-bc4a-4afb-a095-2735738ecc3b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.636586 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778feb3-eeb7-4fe2-9313-4001b4d135d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.637474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778feb3-eeb7-4fe2-9313-4001b4d135d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.637811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778feb3-eeb7-4fe2-9313-4001b4d135d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.638021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778feb3-eeb7-4fe2-9313-4001b4d135d2 admin admin] PATH_INFO: `/v3/projects/b81c4253e5f6406f93505e859266658e/users/2caf5922d8514d3ab14f4d15e72ce625/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.652585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778feb3-eeb7-4fe2-9313-4001b4d135d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81c4253e5f6406f93505e859266658e, user_id=2caf5922d8514d3ab14f4d15e72ce625, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.655265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778feb3-eeb7-4fe2-9313-4001b4d135d2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.670638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 922/1865] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:30 2022] PUT /identity/v3/projects/b81c4253e5f6406f93505e859266658e/users/2caf5922d8514d3ab14f4d15e72ce625/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:30.684529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797f253-4294-4bcc-9a09-b873db34b4fb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.684817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797f253-4294-4bcc-9a09-b873db34b4fb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.685098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797f253-4294-4bcc-9a09-b873db34b4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.715757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6797f253-4294-4bcc-9a09-b873db34b4fb None None] MFA Rules not processed for user `2caf5922d8514d3ab14f4d15e72ce625`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:30.717046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6797f253-4294-4bcc-9a09-b873db34b4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:30.733553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:30.800251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 944/1866] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:30.805410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778feb3-eeb7-4fe2-9313-4001b4d135d2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.876534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778feb3-eeb7-4fe2-9313-4001b4d135d2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:30.883238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095b90-97ea-4e4e-a386-db3eac6fa655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:30.883969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095b90-97ea-4e4e-a386-db3eac6fa655 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:30.884264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095b90-97ea-4e4e-a386-db3eac6fa655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:30.884528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095b90-97ea-4e4e-a386-db3eac6fa655 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:30.885175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49095b90-97ea-4e4e-a386-db3eac6fa655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:30.887319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49095b90-97ea-4e4e-a386-db3eac6fa655 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:30.938661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 923/1867] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:30 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:30.942767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797f253-4294-4bcc-9a09-b873db34b4fb None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:30.953745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797f253-4294-4bcc-9a09-b873db34b4fb None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:31.018472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763dbc08-5b16-49b4-b328-f0e9e3464d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:31.019126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dbc08-5b16-49b4-b328-f0e9e3464d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:31.019401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dbc08-5b16-49b4-b328-f0e9e3464d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:31.019674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dbc08-5b16-49b4-b328-f0e9e3464d7c admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:31.020334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763dbc08-5b16-49b4-b328-f0e9e3464d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:31.022306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763dbc08-5b16-49b4-b328-f0e9e3464d7c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:31.048432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 945/1868] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:30 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:31.049780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095b90-97ea-4e4e-a386-db3eac6fa655 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:31.060736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095b90-97ea-4e4e-a386-db3eac6fa655 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:31.065788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb302e-a74f-48f1-a6c3-cb34bc81cc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:31.066405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb302e-a74f-48f1-a6c3-cb34bc81cc90 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:31.066704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb302e-a74f-48f1-a6c3-cb34bc81cc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:31.067032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb302e-a74f-48f1-a6c3-cb34bc81cc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:31.067684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb302e-a74f-48f1-a6c3-cb34bc81cc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:31.069368 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb302e-a74f-48f1-a6c3-cb34bc81cc90 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:31.081813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 924/1869] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:31.087206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763dbc08-5b16-49b4-b328-f0e9e3464d7c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:31.098832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763dbc08-5b16-49b4-b328-f0e9e3464d7c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:31.104430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8c16e-69d7-48e4-b204-00bf9987feb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:31.105125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c16e-69d7-48e4-b204-00bf9987feb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:31.105446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c16e-69d7-48e4-b204-00bf9987feb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:31.105827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c16e-69d7-48e4-b204-00bf9987feb7 admin admin] PATH_INFO: `/v3/projects/a45f03451c974c049e1277e9fd743a6c/users/508d96e3b2d24a4596d7769ebbfb39f7/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:31.123205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8c16e-69d7-48e4-b204-00bf9987feb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45f03451c974c049e1277e9fd743a6c, user_id=508d96e3b2d24a4596d7769ebbfb39f7, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:31.126172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8c16e-69d7-48e4-b204-00bf9987feb7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:31.140878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 946/1870] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:31 2022] PUT /identity/v3/projects/a45f03451c974c049e1277e9fd743a6c/users/508d96e3b2d24a4596d7769ebbfb39f7/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:31.145075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb302e-a74f-48f1-a6c3-cb34bc81cc90 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:31.155464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb302e-a74f-48f1-a6c3-cb34bc81cc90 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:31.210189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028723e-aecd-4618-9da5-9a5f6f20494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:31.210958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028723e-aecd-4618-9da5-9a5f6f20494c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:31.211244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028723e-aecd-4618-9da5-9a5f6f20494c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:31.211509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028723e-aecd-4618-9da5-9a5f6f20494c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:31.212141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028723e-aecd-4618-9da5-9a5f6f20494c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:31.213734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028723e-aecd-4618-9da5-9a5f6f20494c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:31.223639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 925/1871] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:31.228212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8c16e-69d7-48e4-b204-00bf9987feb7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:31.238937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8c16e-69d7-48e4-b204-00bf9987feb7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:31.244098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:31.244781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:31.245055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:31.245333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] PATH_INFO: `/v3/projects/a45f03451c974c049e1277e9fd743a6c/users/508d96e3b2d24a4596d7769ebbfb39f7/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:31.247287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45f03451c974c049e1277e9fd743a6c, user_id=508d96e3b2d24a4596d7769ebbfb39f7, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:31.249913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:31.260565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 947/1872] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:31 2022] PUT /identity/v3/projects/a45f03451c974c049e1277e9fd743a6c/users/508d96e3b2d24a4596d7769ebbfb39f7/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:31.271690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739348c0-55ef-4119-89ff-2861c98c220a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:31.271983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739348c0-55ef-4119-89ff-2861c98c220a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:31.272261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739348c0-55ef-4119-89ff-2861c98c220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:31.303436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-739348c0-55ef-4119-89ff-2861c98c220a None None] MFA Rules not processed for user `508d96e3b2d24a4596d7769ebbfb39f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:31.304811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-739348c0-55ef-4119-89ff-2861c98c220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:31.319418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:31.382328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 926/1873] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:31.395052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:31.396919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:31.460444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d3f29-36ed-4b94-b014-fd3bbc1b9328 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:31.464252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222240-f6cd-4d91-9e7c-9f00ec57c940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:31.464866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222240-f6cd-4d91-9e7c-9f00ec57c940 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:31.465141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222240-f6cd-4d91-9e7c-9f00ec57c940 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:31.465409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222240-f6cd-4d91-9e7c-9f00ec57c940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:31.490000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:31.527175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f222240-f6cd-4d91-9e7c-9f00ec57c940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:31.528573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f222240-f6cd-4d91-9e7c-9f00ec57c940 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:31.537979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 948/1874] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:33.104123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbb233-e7cf-42ed-9915-48f77bb1b6cc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:33.105012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbb233-e7cf-42ed-9915-48f77bb1b6cc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:33.105288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbb233-e7cf-42ed-9915-48f77bb1b6cc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:33.106959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 927/1875] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:35:33.113679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3de44-9ead-4e8f-bfa1-92d0b3793aa4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:33.114093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3de44-9ead-4e8f-bfa1-92d0b3793aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:33.114093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3de44-9ead-4e8f-bfa1-92d0b3793aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:33.134854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d6d3de44-9ead-4e8f-bfa1-92d0b3793aa4 None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:33.135942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d6d3de44-9ead-4e8f-bfa1-92d0b3793aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:33.232243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 949/1876] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:35:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:33.254599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbb233-e7cf-42ed-9915-48f77bb1b6cc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:33.257320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4dcbb233-e7cf-42ed-9915-48f77bb1b6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:33.289712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbb233-e7cf-42ed-9915-48f77bb1b6cc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:33.296684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:33.297310 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:33.297586 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:33.297858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:33.308004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:33.309674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:33.322265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 928/1877] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:34.179479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3de44-9ead-4e8f-bfa1-92d0b3793aa4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:34.181003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d6d3de44-9ead-4e8f-bfa1-92d0b3793aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:34.192418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3de44-9ead-4e8f-bfa1-92d0b3793aa4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:34.196086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83190896-0e45-496b-8fd4-4b34160006d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:34.196746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83190896-0e45-496b-8fd4-4b34160006d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:34.196976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83190896-0e45-496b-8fd4-4b34160006d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:34.197247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83190896-0e45-496b-8fd4-4b34160006d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:34.257648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83190896-0e45-496b-8fd4-4b34160006d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:34.259372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83190896-0e45-496b-8fd4-4b34160006d9 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:34.268953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 950/1878] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:35.037553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:35.042410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:35.083069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83190896-0e45-496b-8fd4-4b34160006d9 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:35.097136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83190896-0e45-496b-8fd4-4b34160006d9 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:35.103603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b9250-3b63-4d97-a018-6baf0de802c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:35.104600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b9250-3b63-4d97-a018-6baf0de802c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:35.105322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b9250-3b63-4d97-a018-6baf0de802c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:35.105835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b9250-3b63-4d97-a018-6baf0de802c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:35.129697 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b9250-3b63-4d97-a018-6baf0de802c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:35.131610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b9250-3b63-4d97-a018-6baf0de802c4 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:35.154059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 951/1879] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:35.203949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba923463-e9e0-4b71-b119-f8c709090c14 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:35.221977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:35.224320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:35.225832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:35.226117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] PATH_INFO: `/v3/users/bc2a4e1ca0784a4d954f6805a9c4c47b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:35.230183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2a4e1ca0784a4d954f6805a9c4c47b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:35.242345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:35.327752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2a4e1ca0784a4d954f6805a9c4c47b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:35.334868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2a4e1ca0784a4d954f6805a9c4c47b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:35.335559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2a4e1ca0784a4d954f6805a9c4c47b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:35.348988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2a4e1ca0784a4d954f6805a9c4c47b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:35.349667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2a4e1ca0784a4d954f6805a9c4c47b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:35.361818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 929/1880] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:35 2022] DELETE /identity/v3/users/bc2a4e1ca0784a4d954f6805a9c4c47b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:35.366314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbbbb7-d028-43db-9266-0acd8f6c8a02 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:35.366780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbbbb7-d028-43db-9266-0acd8f6c8a02 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:35.366926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbbbb7-d028-43db-9266-0acd8f6c8a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:35.385970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-73cbbbb7-d028-43db-9266-0acd8f6c8a02 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:35.387337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-73cbbbb7-d028-43db-9266-0acd8f6c8a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:35.424349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:35.450452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 952/1881] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:35.460457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:35.462032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:35.482224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:35.524571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bdb09-20e3-4658-8976-f65876a49347 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:35.528929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdacc4b-5737-4549-b4c8-47f00ee5dfcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:35.529592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdacc4b-5737-4549-b4c8-47f00ee5dfcc service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:35.529867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdacc4b-5737-4549-b4c8-47f00ee5dfcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:35.530166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdacc4b-5737-4549-b4c8-47f00ee5dfcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:35.540328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdacc4b-5737-4549-b4c8-47f00ee5dfcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:35.541814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdacc4b-5737-4549-b4c8-47f00ee5dfcc service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:35.551588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 930/1882] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:35.912952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbbbb7-d028-43db-9266-0acd8f6c8a02 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:35.914517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-73cbbbb7-d028-43db-9266-0acd8f6c8a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:35.926511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbbbb7-d028-43db-9266-0acd8f6c8a02 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:35.931694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:35.933014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:35.933434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:35.933843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] PATH_INFO: `/v3/projects/0b68ad5fb6c546619bb5b18f6358de78` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:35.935369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b68ad5fb6c546619bb5b18f6358de78)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:35.941124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:36.016832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b68ad5fb6c546619bb5b18f6358de78', 'request_id': 'req-8d2c753b-e572-4c4f-811d-4341c257b043'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.030034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b68ad5fb6c546619bb5b18f6358de78', 'request_id': 'req-8d2c753b-e572-4c4f-811d-4341c257b043'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.042254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b68ad5fb6c546619bb5b18f6358de78', 'request_id': 'req-8d2c753b-e572-4c4f-811d-4341c257b043'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.057019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] The token cache is being invalidate because project 0b68ad5fb6c546619bb5b18f6358de78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:36.057410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.058203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.059892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 953/1883] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:35 2022] DELETE /identity/v3/projects/0b68ad5fb6c546619bb5b18f6358de78 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:36.064610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdacc4b-5737-4549-b4c8-47f00ee5dfcc service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:36.081829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdacc4b-5737-4549-b4c8-47f00ee5dfcc service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:36.172242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:36.175341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:36.176856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:36.177217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] PATH_INFO: `/v3/users/3be9983e926743bcacb402b291c89d74` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:36.179666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be9983e926743bcacb402b291c89d74)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:36.182019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:36.288813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be9983e926743bcacb402b291c89d74'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.323549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be9983e926743bcacb402b291c89d74'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.324477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be9983e926743bcacb402b291c89d74'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.379990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be9983e926743bcacb402b291c89d74'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.380700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be9983e926743bcacb402b291c89d74'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:36.421305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 931/1884] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:36 2022] DELETE /identity/v3/users/3be9983e926743bcacb402b291c89d74 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:37.154209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.157124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:37.174802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c753b-e572-4c4f-811d-4341c257b043 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.294915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.295675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.295965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.296236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] PATH_INFO: `/v3/projects/1de531046e80469db3fb9c3c7c494240` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.297513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de531046e80469db3fb9c3c7c494240)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.299541 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.346768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.350119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:37.367044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd919ee1-1a4a-4c55-8d2d-3c3afc013bae admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.369004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de531046e80469db3fb9c3c7c494240', 'request_id': 'req-30b43b83-f75a-4695-aa5a-67ecee268621'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.379831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de531046e80469db3fb9c3c7c494240', 'request_id': 'req-30b43b83-f75a-4695-aa5a-67ecee268621'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.388563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de531046e80469db3fb9c3c7c494240', 'request_id': 'req-30b43b83-f75a-4695-aa5a-67ecee268621'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.393107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:37.408573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] The token cache is being invalidate because project 1de531046e80469db3fb9c3c7c494240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:37.408751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.409511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-30b43b83-f75a-4695-aa5a-67ecee268621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.411671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 954/1885] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:37 2022] DELETE /identity/v3/projects/1de531046e80469db3fb9c3c7c494240 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:37.417229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec1e7-8725-41dd-8e9a-6647d1316264 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.417517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec1e7-8725-41dd-8e9a-6647d1316264 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.417787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec1e7-8725-41dd-8e9a-6647d1316264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.437885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:37.451021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-84aec1e7-8725-41dd-8e9a-6647d1316264 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:37.454572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-84aec1e7-8725-41dd-8e9a-6647d1316264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:37.484601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.485269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.485748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.486299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] PATH_INFO: `/v3/users/6d9c94625d8f4237b3ea8667ebc22259` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.487523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d9c94625d8f4237b3ea8667ebc22259)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.489644 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.541543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 955/1886] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:37.550255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f1f1-87c3-453f-a4ce-d5552205674b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.550669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f1f1-87c3-453f-a4ce-d5552205674b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.551083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f1f1-87c3-453f-a4ce-d5552205674b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.554193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d9c94625d8f4237b3ea8667ebc22259'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.564040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d9c94625d8f4237b3ea8667ebc22259'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.564763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d9c94625d8f4237b3ea8667ebc22259'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.574681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-98e8f1f1-87c3-453f-a4ce-d5552205674b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:37.576070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-98e8f1f1-87c3-453f-a4ce-d5552205674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:37.590797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 956/1887] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:37.591534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8f1f1-87c3-453f-a4ce-d5552205674b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.596483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d9c94625d8f4237b3ea8667ebc22259'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.597211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5c80662-634b-4695-9fb5-da7af5444ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d9c94625d8f4237b3ea8667ebc22259'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:37.601838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8f1f1-87c3-453f-a4ce-d5552205674b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.607054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.607779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.608070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.608324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.609035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:37.609335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.611051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.611941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 932/1888] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:37 2022] DELETE /identity/v3/users/6d9c94625d8f4237b3ea8667ebc22259 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:37.616382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cea4f-8130-44f2-9012-fa737bc0728b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.616663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cea4f-8130-44f2-9012-fa737bc0728b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.617024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cea4f-8130-44f2-9012-fa737bc0728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.626738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 957/1889] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:35:37.627726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.629354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:37.637201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-726cea4f-8130-44f2-9012-fa737bc0728b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:37.638377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-726cea4f-8130-44f2-9012-fa737bc0728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:37.641895 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f624d4-a176-4677-90d2-0cf341bae559 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.703148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10b28c-1aeb-48a2-9bd0-ab4e3f98ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.703818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b28c-1aeb-48a2-9bd0-ab4e3f98ec7f admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.704459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b28c-1aeb-48a2-9bd0-ab4e3f98ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.704459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b28c-1aeb-48a2-9bd0-ab4e3f98ec7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.704997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10b28c-1aeb-48a2-9bd0-ab4e3f98ec7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.706328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 933/1890] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:37.707133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10b28c-1aeb-48a2-9bd0-ab4e3f98ec7f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.717169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cea4f-8130-44f2-9012-fa737bc0728b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.718550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-726cea4f-8130-44f2-9012-fa737bc0728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:37.737286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 958/1891] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:37 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:37.741751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10b28c-1aeb-48a2-9bd0-ab4e3f98ec7f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.778485 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cea4f-8130-44f2-9012-fa737bc0728b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.787751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:37.804996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10b28c-1aeb-48a2-9bd0-ab4e3f98ec7f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.809980 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc52301-7f7a-4e89-b1b1-faa5ed039593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.810596 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52301-7f7a-4e89-b1b1-faa5ed039593 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.810900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52301-7f7a-4e89-b1b1-faa5ed039593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.811170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52301-7f7a-4e89-b1b1-faa5ed039593 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.811867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc52301-7f7a-4e89-b1b1-faa5ed039593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.813761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc52301-7f7a-4e89-b1b1-faa5ed039593 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.821979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:37.833195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7311d-8f76-4778-98eb-b523499ba4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.833823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7311d-8f76-4778-98eb-b523499ba4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.834100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7311d-8f76-4778-98eb-b523499ba4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.834370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7311d-8f76-4778-98eb-b523499ba4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.838474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 959/1892] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:37 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:37.842396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc52301-7f7a-4e89-b1b1-faa5ed039593 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.845756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7311d-8f76-4778-98eb-b523499ba4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.847232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7311d-8f76-4778-98eb-b523499ba4f3 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.852728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc52301-7f7a-4e89-b1b1-faa5ed039593 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.857106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 934/1893] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:35:37.857726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30629a7b-e208-40dd-90e1-3de1717a532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.858597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30629a7b-e208-40dd-90e1-3de1717a532f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.858916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30629a7b-e208-40dd-90e1-3de1717a532f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.859210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30629a7b-e208-40dd-90e1-3de1717a532f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.859889 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30629a7b-e208-40dd-90e1-3de1717a532f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.861539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30629a7b-e208-40dd-90e1-3de1717a532f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.875669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 960/1894] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:37.876932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7311d-8f76-4778-98eb-b523499ba4f3 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.887143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7311d-8f76-4778-98eb-b523499ba4f3 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.892364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8eec9d-f033-4001-a331-4e27861607c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.893112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eec9d-f033-4001-a331-4e27861607c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.893391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eec9d-f033-4001-a331-4e27861607c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.893657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eec9d-f033-4001-a331-4e27861607c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.894432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8eec9d-f033-4001-a331-4e27861607c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.896066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8eec9d-f033-4001-a331-4e27861607c4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.910512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 935/1895] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:37.912129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30629a7b-e208-40dd-90e1-3de1717a532f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:37.926070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30629a7b-e208-40dd-90e1-3de1717a532f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:37.933765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea78cd0-36c5-4f3a-b430-656b44b2fbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:37.934478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea78cd0-36c5-4f3a-b430-656b44b2fbb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.934790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea78cd0-36c5-4f3a-b430-656b44b2fbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.935073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea78cd0-36c5-4f3a-b430-656b44b2fbb4 admin admin] PATH_INFO: `/v3/projects/03af52ca13314c74b06ef1db681cea39/users/5f4ec9e916fb42b1adbcbeab0fe75eea/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:37.957562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea78cd0-36c5-4f3a-b430-656b44b2fbb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03af52ca13314c74b06ef1db681cea39, user_id=5f4ec9e916fb42b1adbcbeab0fe75eea, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:37.961377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea78cd0-36c5-4f3a-b430-656b44b2fbb4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:37.980359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 961/1896] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:37 2022] PUT /identity/v3/projects/03af52ca13314c74b06ef1db681cea39/users/5f4ec9e916fb42b1adbcbeab0fe75eea/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:37.991160 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef4bd3-e7e6-4b7b-941a-8034bf95cab3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:37.991280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef4bd3-e7e6-4b7b-941a-8034bf95cab3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:37.991539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef4bd3-e7e6-4b7b-941a-8034bf95cab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:38.025903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4cef4bd3-e7e6-4b7b-941a-8034bf95cab3 None None] MFA Rules not processed for user `5f4ec9e916fb42b1adbcbeab0fe75eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:38.027224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4cef4bd3-e7e6-4b7b-941a-8034bf95cab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:38.146776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 936/1897] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:38.148673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea78cd0-36c5-4f3a-b430-656b44b2fbb4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:38.254182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea78cd0-36c5-4f3a-b430-656b44b2fbb4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:38.255743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef4bd3-e7e6-4b7b-941a-8034bf95cab3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:38.257160 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4cef4bd3-e7e6-4b7b-941a-8034bf95cab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:38.267922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62362727-e58b-414a-86d4-59ad89bf0178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:38.268672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62362727-e58b-414a-86d4-59ad89bf0178 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:38.268839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62362727-e58b-414a-86d4-59ad89bf0178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:38.269156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62362727-e58b-414a-86d4-59ad89bf0178 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:38.269745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62362727-e58b-414a-86d4-59ad89bf0178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:38.271724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62362727-e58b-414a-86d4-59ad89bf0178 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:38.286318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef4bd3-e7e6-4b7b-941a-8034bf95cab3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:38.294991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:38.296103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:38.301118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:38.301988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] PATH_INFO: `/v3/projects/9dbade89d75a4cd39a77873b1190e072` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:38.303053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dbade89d75a4cd39a77873b1190e072)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:38.305333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:38.326592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 962/1898] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:38 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:38.331154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62362727-e58b-414a-86d4-59ad89bf0178 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:38.412253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dbade89d75a4cd39a77873b1190e072', 'request_id': 'req-eb36e624-595e-4680-b087-a01408b7d413'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.420021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dbade89d75a4cd39a77873b1190e072', 'request_id': 'req-eb36e624-595e-4680-b087-a01408b7d413'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.435935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dbade89d75a4cd39a77873b1190e072', 'request_id': 'req-eb36e624-595e-4680-b087-a01408b7d413'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.457382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] The token cache is being invalidate because project 9dbade89d75a4cd39a77873b1190e072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:38.457733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.458463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.460408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 937/1899] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:38 2022] DELETE /identity/v3/projects/9dbade89d75a4cd39a77873b1190e072 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:38.464500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:38.466009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:38.493775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62362727-e58b-414a-86d4-59ad89bf0178 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:38.526844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:38.648358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:38.669506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36e624-595e-4680-b087-a01408b7d413 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:38.677109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:38.677802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:38.678104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:38.682205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:38.682984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:38.687179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:38.692503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:38.693170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:38.693446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:38.693748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] PATH_INFO: `/v3/users/1617f1e5439342ae8177eb2f5f815df5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:38.695449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1617f1e5439342ae8177eb2f5f815df5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:38.697516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:38.755164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 963/1900] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:38 2022] POST /identity/v3/users => generated 297 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:38.756927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:38.759229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:38.777948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dc924-ce82-45be-bb91-5751f2fb12ca admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:38.783698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074b3e-f241-4122-b7a3-bb541afea631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:38.784317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074b3e-f241-4122-b7a3-bb541afea631 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:38.784615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074b3e-f241-4122-b7a3-bb541afea631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:38.785348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074b3e-f241-4122-b7a3-bb541afea631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:38.785991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1617f1e5439342ae8177eb2f5f815df5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.786104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074b3e-f241-4122-b7a3-bb541afea631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:38.787650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074b3e-f241-4122-b7a3-bb541afea631 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:38.799506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 964/1901] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:38.799904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1617f1e5439342ae8177eb2f5f815df5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.800831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1617f1e5439342ae8177eb2f5f815df5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.803653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074b3e-f241-4122-b7a3-bb541afea631 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:38.816717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074b3e-f241-4122-b7a3-bb541afea631 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:38.820164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1617f1e5439342ae8177eb2f5f815df5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.821083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1617f1e5439342ae8177eb2f5f815df5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:38.822059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79e8a-95a8-4ef7-88ae-8ba8f9a26f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:38.822825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79e8a-95a8-4ef7-88ae-8ba8f9a26f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:38.823164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79e8a-95a8-4ef7-88ae-8ba8f9a26f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:38.823540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79e8a-95a8-4ef7-88ae-8ba8f9a26f30 admin admin] PATH_INFO: `/v3/projects/f7db5c3cf55e463f9b0b370839c96b90/users/9ff904d759434a878c32a927958ef57d/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:38.841464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 938/1902] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:38 2022] DELETE /identity/v3/users/1617f1e5439342ae8177eb2f5f815df5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:38.842084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee79e8a-95a8-4ef7-88ae-8ba8f9a26f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7db5c3cf55e463f9b0b370839c96b90, user_id=9ff904d759434a878c32a927958ef57d, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:38.844863 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee79e8a-95a8-4ef7-88ae-8ba8f9a26f30 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:38.863062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 965/1903] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:38 2022] PUT /identity/v3/projects/f7db5c3cf55e463f9b0b370839c96b90/users/9ff904d759434a878c32a927958ef57d/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:38.867158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:38.868987 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:38.953814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c0dbe-8958-420e-884f-ec6f7c70f552 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:39.007070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:39.026098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f3448-9e2f-4bba-a013-bff9ff128367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:39.026915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f3448-9e2f-4bba-a013-bff9ff128367 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:39.027204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f3448-9e2f-4bba-a013-bff9ff128367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:39.027466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f3448-9e2f-4bba-a013-bff9ff128367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:39.028129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f3448-9e2f-4bba-a013-bff9ff128367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:39.030181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f3448-9e2f-4bba-a013-bff9ff128367 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:39.041754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 939/1904] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:35:39.046105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79e8a-95a8-4ef7-88ae-8ba8f9a26f30 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:39.057475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79e8a-95a8-4ef7-88ae-8ba8f9a26f30 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:39.078505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:39.079613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:39.079891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:39.080509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] PATH_INFO: `/v3/projects/f7db5c3cf55e463f9b0b370839c96b90/users/9ff904d759434a878c32a927958ef57d/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:39.083265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7db5c3cf55e463f9b0b370839c96b90, user_id=9ff904d759434a878c32a927958ef57d, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:39.086740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:39.106473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 966/1905] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:39 2022] PUT /identity/v3/projects/f7db5c3cf55e463f9b0b370839c96b90/users/9ff904d759434a878c32a927958ef57d/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:39.119267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2879-be8f-41bd-8758-a0ff337e5c5e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:39.119529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2879-be8f-41bd-8758-a0ff337e5c5e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:39.119786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2879-be8f-41bd-8758-a0ff337e5c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:39.150416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-233a2879-be8f-41bd-8758-a0ff337e5c5e None None] MFA Rules not processed for user `9ff904d759434a878c32a927958ef57d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:39.151666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-233a2879-be8f-41bd-8758-a0ff337e5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:39.235826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 940/1906] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:39.248332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:39.249995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:39.313724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a9e53-3882-4270-91ae-9a23193edde1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:39.317712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade6adb-5f26-4dae-99db-eb1e91f15103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:39.318452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6adb-5f26-4dae-99db-eb1e91f15103 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:39.318746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6adb-5f26-4dae-99db-eb1e91f15103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:39.319036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6adb-5f26-4dae-99db-eb1e91f15103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:39.339252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade6adb-5f26-4dae-99db-eb1e91f15103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:39.340773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade6adb-5f26-4dae-99db-eb1e91f15103 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:39.352096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 967/1907] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:39.473728 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a2879-be8f-41bd-8758-a0ff337e5c5e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:39.484046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a2879-be8f-41bd-8758-a0ff337e5c5e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:39.487903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:39.488519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:39.488782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:39.489050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:39.490841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:39.552007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d59f7-5796-4bf5-b8dc-ca32f50a729e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:39.552364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d59f7-5796-4bf5-b8dc-ca32f50a729e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:39.552980 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d59f7-5796-4bf5-b8dc-ca32f50a729e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:39.554754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 968/1908] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:35:39.566689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:39.568165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:39.577135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 941/1909] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:39.592506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e888b-7f9a-46da-920c-bb70ff37b519 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:39.592790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e888b-7f9a-46da-920c-bb70ff37b519 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:39.593058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e888b-7f9a-46da-920c-bb70ff37b519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:39.629246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-448e888b-7f9a-46da-920c-bb70ff37b519 None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:39.631367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-448e888b-7f9a-46da-920c-bb70ff37b519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:39.732868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:39.735127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:39.794192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 969/1910] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:35:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:39.807765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e888b-7f9a-46da-920c-bb70ff37b519 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:39.809554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-448e888b-7f9a-46da-920c-bb70ff37b519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:39.876129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:39.884829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f4df5-afbd-4f0c-ba51-1bb6bce5dbac service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:39.894784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:39.895677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:39.896095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:39.896536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] PATH_INFO: `/v3/projects/200c15da9b804a188712f2874ca07393` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:39.899675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200c15da9b804a188712f2874ca07393)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:39.902178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:39.914083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:39.967621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e888b-7f9a-46da-920c-bb70ff37b519 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:39.973692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200c15da9b804a188712f2874ca07393', 'request_id': 'req-cae56f3c-5589-4615-8242-47de8cea2e87'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:39.986039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200c15da9b804a188712f2874ca07393', 'request_id': 'req-cae56f3c-5589-4615-8242-47de8cea2e87'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.027689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200c15da9b804a188712f2874ca07393', 'request_id': 'req-cae56f3c-5589-4615-8242-47de8cea2e87'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.050351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] The token cache is being invalidate because project 200c15da9b804a188712f2874ca07393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:40.050676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.051342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cae56f3c-5589-4615-8242-47de8cea2e87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.052942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 942/1911] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:39 2022] DELETE /identity/v3/projects/200c15da9b804a188712f2874ca07393 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:40.054139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6fc6-8fb9-495f-af59-b7482fde85c3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.054418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6fc6-8fb9-495f-af59-b7482fde85c3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.054682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6fc6-8fb9-495f-af59-b7482fde85c3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.056190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 943/1912] 10.222.0.42 () {54 vars in 961 bytes} [Wed Aug 31 04:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:35:40.057491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad6fc6-8fb9-495f-af59-b7482fde85c3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:40.062779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88ad6fc6-8fb9-495f-af59-b7482fde85c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.078201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:40.078840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.079123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.079443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] PATH_INFO: `/v3/users/81ff6bd2f3da46378e5cb1ff956e70d6` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.080702 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ff6bd2f3da46378e5cb1ff956e70d6)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:40.082650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:40.141032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ff6bd2f3da46378e5cb1ff956e70d6'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.146912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad6fc6-8fb9-495f-af59-b7482fde85c3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:40.151190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ff6bd2f3da46378e5cb1ff956e70d6'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.152046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ff6bd2f3da46378e5cb1ff956e70d6'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.152290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:40.152933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.153384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.153660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] PATH_INFO: `/v3/users/5f4ec9e916fb42b1adbcbeab0fe75eea` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.155274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f4ec9e916fb42b1adbcbeab0fe75eea)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:40.157166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:40.166537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ff6bd2f3da46378e5cb1ff956e70d6'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.167289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b3f457c1-cb24-4777-8a87-c926dcc643a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ff6bd2f3da46378e5cb1ff956e70d6'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.183072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 970/1913] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:39 2022] DELETE /identity/v3/users/81ff6bd2f3da46378e5cb1ff956e70d6 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:40.184629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083cf0-2717-4cb0-800b-b58a4a302959 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.184629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083cf0-2717-4cb0-800b-b58a4a302959 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.184925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083cf0-2717-4cb0-800b-b58a4a302959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.209717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-58083cf0-2717-4cb0-800b-b58a4a302959 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:40.210972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-58083cf0-2717-4cb0-800b-b58a4a302959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.229260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f4ec9e916fb42b1adbcbeab0fe75eea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.243570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f4ec9e916fb42b1adbcbeab0fe75eea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.244675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f4ec9e916fb42b1adbcbeab0fe75eea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.264408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f4ec9e916fb42b1adbcbeab0fe75eea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.265477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-858d35a5-1f44-4a79-a3c5-e8e311944c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f4ec9e916fb42b1adbcbeab0fe75eea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:40.287459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 944/1914] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:40 2022] DELETE /identity/v3/users/5f4ec9e916fb42b1adbcbeab0fe75eea => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:40.287459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8add-7c72-4a7b-b759-38ef33e461f8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.287459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8add-7c72-4a7b-b759-38ef33e461f8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.287745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:40.287857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8add-7c72-4a7b-b759-38ef33e461f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.317222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 971/1915] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:40.318504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731653a-37a0-4034-8508-c7654acd52b6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.318915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731653a-37a0-4034-8508-c7654acd52b6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.319292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731653a-37a0-4034-8508-c7654acd52b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.324737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5cdd8add-7c72-4a7b-b759-38ef33e461f8 None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:40.327947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5cdd8add-7c72-4a7b-b759-38ef33e461f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.344619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0731653a-37a0-4034-8508-c7654acd52b6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:40.346759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0731653a-37a0-4034-8508-c7654acd52b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.352502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:40.423097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 972/1916] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:40.424908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009e0e7-881a-4b15-8a80-713e04b6501b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.425256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009e0e7-881a-4b15-8a80-713e04b6501b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.425613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009e0e7-881a-4b15-8a80-713e04b6501b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.430365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 945/1917] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Aug 31 04:35:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:40.432849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9afe1-4811-4d03-b82f-9efb4db50d79 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.433293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9afe1-4811-4d03-b82f-9efb4db50d79 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.433856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9afe1-4811-4d03-b82f-9efb4db50d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.448239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6009e0e7-881a-4b15-8a80-713e04b6501b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:40.450278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6009e0e7-881a-4b15-8a80-713e04b6501b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.463040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 973/1918] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:40.463040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009e0e7-881a-4b15-8a80-713e04b6501b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:40.490930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009e0e7-881a-4b15-8a80-713e04b6501b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:40.491021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0fd9afe1-4811-4d03-b82f-9efb4db50d79 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:40.491438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:40.493308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0fd9afe1-4811-4d03-b82f-9efb4db50d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.493424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.493424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.493424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.495431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.507392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 946/1919] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:40.508599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9afe1-4811-4d03-b82f-9efb4db50d79 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:40.508708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:40.510285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:40.511815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0fd9afe1-4811-4d03-b82f-9efb4db50d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.521040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 974/1920] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:40.521965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:40.523470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:40.614446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9afe1-4811-4d03-b82f-9efb4db50d79 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:40.618994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a198be2-0ed7-4f8a-814d-adebad389995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:40.620156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a198be2-0ed7-4f8a-814d-adebad389995 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.620537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a198be2-0ed7-4f8a-814d-adebad389995 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.621194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a198be2-0ed7-4f8a-814d-adebad389995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.625864 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436638c8-79d8-4bc0-a827-37e0154092da service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:40.632101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a198be2-0ed7-4f8a-814d-adebad389995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:40.632250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a041d-4438-41b1-a60e-e714dc7a9965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:40.632859 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a041d-4438-41b1-a60e-e714dc7a9965 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.633339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a041d-4438-41b1-a60e-e714dc7a9965 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.633902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a198be2-0ed7-4f8a-814d-adebad389995 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:40.634261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a041d-4438-41b1-a60e-e714dc7a9965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.649516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a041d-4438-41b1-a60e-e714dc7a9965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:40.651240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a041d-4438-41b1-a60e-e714dc7a9965 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:40.652378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 947/1921] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:40.653277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a198be2-0ed7-4f8a-814d-adebad389995 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:40.663917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a198be2-0ed7-4f8a-814d-adebad389995 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:40.668740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:40.669347 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.669630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.669896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.670817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:40.671117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:40.672862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:40.687765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 975/1922] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:40.694916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 948/1923] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:35:40.700083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a041d-4438-41b1-a60e-e714dc7a9965 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:40.714802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a041d-4438-41b1-a60e-e714dc7a9965 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:40.721929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af9b81-66e4-4315-aaae-f87ad8395b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:40.722544 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9b81-66e4-4315-aaae-f87ad8395b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.722862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9b81-66e4-4315-aaae-f87ad8395b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.723132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9b81-66e4-4315-aaae-f87ad8395b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.725149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af9b81-66e4-4315-aaae-f87ad8395b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:40.727533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af9b81-66e4-4315-aaae-f87ad8395b4e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:40.775541 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 976/1924] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:40 2022] POST /identity/v3/projects => generated 357 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:40.780075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:40.809433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a88150-fb99-4dc6-b1ff-f5d3db680156 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:40.899984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e45202-7194-4541-b7c8-3f153c1fc4fc None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.900264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e45202-7194-4541-b7c8-3f153c1fc4fc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.900539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e45202-7194-4541-b7c8-3f153c1fc4fc None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.902852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 977/1925] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:35:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:35:40.947121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae73bb-c0f4-420a-a674-def301193742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:40.947121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae73bb-c0f4-420a-a674-def301193742 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:40.947121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae73bb-c0f4-420a-a674-def301193742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:40.947121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae73bb-c0f4-420a-a674-def301193742 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:40.947458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae73bb-c0f4-420a-a674-def301193742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:40.949599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae73bb-c0f4-420a-a674-def301193742 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:40.979286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 949/1926] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:40 2022] POST /identity/v3/users => generated 279 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:40.983183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e45202-7194-4541-b7c8-3f153c1fc4fc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.008010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e45202-7194-4541-b7c8-3f153c1fc4fc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.014235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcdd64-9a77-4300-b45c-4ffaf61fabf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.014813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcdd64-9a77-4300-b45c-4ffaf61fabf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.015101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcdd64-9a77-4300-b45c-4ffaf61fabf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.015504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcdd64-9a77-4300-b45c-4ffaf61fabf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.016301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcdd64-9a77-4300-b45c-4ffaf61fabf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.018367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcdd64-9a77-4300-b45c-4ffaf61fabf9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.031550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 978/1927] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:41.036336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae73bb-c0f4-420a-a674-def301193742 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.051067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d2669-d5f4-4293-9f63-129c1a487410 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.053472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d2669-d5f4-4293-9f63-129c1a487410 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.053967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d2669-d5f4-4293-9f63-129c1a487410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.054783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae73bb-c0f4-420a-a674-def301193742 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.079692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca8f94-3451-497d-a4eb-5c73e0e41187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.080640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca8f94-3451-497d-a4eb-5c73e0e41187 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.081286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca8f94-3451-497d-a4eb-5c73e0e41187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.084506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca8f94-3451-497d-a4eb-5c73e0e41187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.090082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b83d2669-d5f4-4293-9f63-129c1a487410 None None] MFA Rules not processed for user `463279871f78480d9fee052518539b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:41.091057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca8f94-3451-497d-a4eb-5c73e0e41187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.091576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b83d2669-d5f4-4293-9f63-129c1a487410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:41.095052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca8f94-3451-497d-a4eb-5c73e0e41187 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.125785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 950/1928] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:41.130529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca8f94-3451-497d-a4eb-5c73e0e41187 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.149073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:41.149888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca8f94-3451-497d-a4eb-5c73e0e41187 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.155513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb68fa6-c8a5-4636-9803-39140f26ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.156141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb68fa6-c8a5-4636-9803-39140f26ee6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.156434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb68fa6-c8a5-4636-9803-39140f26ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.156705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb68fa6-c8a5-4636-9803-39140f26ee6a admin admin] PATH_INFO: `/v3/projects/99e46fc8e45b4ef685c2a95aca3b61c6/users/54edc7e36be14e3382c757c4e1d651fd/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.170931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:41.190127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb68fa6-c8a5-4636-9803-39140f26ee6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e46fc8e45b4ef685c2a95aca3b61c6, user_id=54edc7e36be14e3382c757c4e1d651fd, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.196324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 979/1929] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:35:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:41.197303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d2669-d5f4-4293-9f63-129c1a487410 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.200119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b83d2669-d5f4-4293-9f63-129c1a487410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:41.201952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb68fa6-c8a5-4636-9803-39140f26ee6a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.234837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 951/1930] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:41 2022] PUT /identity/v3/projects/99e46fc8e45b4ef685c2a95aca3b61c6/users/54edc7e36be14e3382c757c4e1d651fd/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:41.234837 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb68fa6-c8a5-4636-9803-39140f26ee6a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.238422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d2669-d5f4-4293-9f63-129c1a487410 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.355160 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb68fa6-c8a5-4636-9803-39140f26ee6a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.359166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.361506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.362728 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.363048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.363519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] PATH_INFO: `/v3/users/6a11c5a47a7d48528916c1fc6a6e681e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.363787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.364069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.364375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.366397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a11c5a47a7d48528916c1fc6a6e681e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.370103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.376264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.378095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.390774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 952/1931] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:41.391639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.392951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:41.411982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bc17f-801d-4604-b4bc-6a9cf5a231a6 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.424346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.425208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.425500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.425873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] PATH_INFO: `/v3/projects/03af52ca13314c74b06ef1db681cea39` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.427288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03af52ca13314c74b06ef1db681cea39)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.430416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.441561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a11c5a47a7d48528916c1fc6a6e681e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.454027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a11c5a47a7d48528916c1fc6a6e681e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.455187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a11c5a47a7d48528916c1fc6a6e681e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.474864 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a11c5a47a7d48528916c1fc6a6e681e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.475799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9bac43c0-9233-43b2-80fc-041c0a6e3fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a11c5a47a7d48528916c1fc6a6e681e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.490474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 980/1932] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:41 2022] DELETE /identity/v3/users/6a11c5a47a7d48528916c1fc6a6e681e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:41.491914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e07a9-fc0b-4c4a-b9e6-185ff719a528 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.492195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e07a9-fc0b-4c4a-b9e6-185ff719a528 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.492494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e07a9-fc0b-4c4a-b9e6-185ff719a528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.507592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03af52ca13314c74b06ef1db681cea39', 'request_id': 'req-335305d0-a576-40e8-9d27-97ecdd4863f5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.517042 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03af52ca13314c74b06ef1db681cea39', 'request_id': 'req-335305d0-a576-40e8-9d27-97ecdd4863f5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.524543 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-048e07a9-fc0b-4c4a-b9e6-185ff719a528 None None] MFA Rules not processed for user `54edc7e36be14e3382c757c4e1d651fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:41.526124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-048e07a9-fc0b-4c4a-b9e6-185ff719a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:41.529847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03af52ca13314c74b06ef1db681cea39', 'request_id': 'req-335305d0-a576-40e8-9d27-97ecdd4863f5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.548120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] The token cache is being invalidate because project 03af52ca13314c74b06ef1db681cea39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:41.548412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.551622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.551622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 953/1933] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:41 2022] DELETE /identity/v3/projects/03af52ca13314c74b06ef1db681cea39 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:41.551622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.552982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:41.578631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:41.602640 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 981/1934] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:41.603504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e07a9-fc0b-4c4a-b9e6-185ff719a528 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.604862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-048e07a9-fc0b-4c4a-b9e6-185ff719a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:41.635768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335305d0-a576-40e8-9d27-97ecdd4863f5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.640054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:41.687281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e07a9-fc0b-4c4a-b9e6-185ff719a528 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.696240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.697098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.697454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.697801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] PATH_INFO: `/v3/users/9ff904d759434a878c32a927958ef57d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.699318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ff904d759434a878c32a927958ef57d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.701399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.719230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.719885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.720153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.720780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] PATH_INFO: `/v3/projects/0b2788168e1b444fb9d8ab7d3f986371` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.722143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2788168e1b444fb9d8ab7d3f986371)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.724073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.759951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ff904d759434a878c32a927958ef57d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.769249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ff904d759434a878c32a927958ef57d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.769892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ff904d759434a878c32a927958ef57d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.783509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ff904d759434a878c32a927958ef57d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.783906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2788168e1b444fb9d8ab7d3f986371', 'request_id': 'req-c5b3dbda-ad25-48fd-a56c-a330c432886e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.784213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ff904d759434a878c32a927958ef57d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.790761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2788168e1b444fb9d8ab7d3f986371', 'request_id': 'req-c5b3dbda-ad25-48fd-a56c-a330c432886e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.797057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 982/1935] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:41 2022] DELETE /identity/v3/users/9ff904d759434a878c32a927958ef57d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:41.797956 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.799393 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:41.801805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2788168e1b444fb9d8ab7d3f986371', 'request_id': 'req-c5b3dbda-ad25-48fd-a56c-a330c432886e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.818022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] The token cache is being invalidate because project 0b2788168e1b444fb9d8ab7d3f986371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:41.818409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.819331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:41.821149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 954/1936] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:41 2022] DELETE /identity/v3/projects/0b2788168e1b444fb9d8ab7d3f986371 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:41.822013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:41.823787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:41.950395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eade65-2208-45ee-9c87-f87e9357ea10 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.955155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3dbda-ad25-48fd-a56c-a330c432886e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:41.960573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.961306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.961680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.961825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] PATH_INFO: `/v3/projects/365e83c8647b401da0040ff09e9cda7e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.963399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365e83c8647b401da0040ff09e9cda7e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.965331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:41.977512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:41.978402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:41.979294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:41.980394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:41.984403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:41.987006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:42.007217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:42.045749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 983/1937] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:41 2022] POST /identity/v3/projects => generated 357 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:42.046820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.053298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:42.055670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:42.076248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365e83c8647b401da0040ff09e9cda7e', 'request_id': 'req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.083682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365e83c8647b401da0040ff09e9cda7e', 'request_id': 'req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.096242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365e83c8647b401da0040ff09e9cda7e', 'request_id': 'req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.111734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] The token cache is being invalidate because project 365e83c8647b401da0040ff09e9cda7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:42.112049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.112780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.117531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 955/1938] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:41 2022] DELETE /identity/v3/projects/365e83c8647b401da0040ff09e9cda7e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:42.117531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.121640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:42.179707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52653d55-4a66-4a68-bae9-774e6d7bb502 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:42.190147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:42.191030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.192032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.192312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] PATH_INFO: `/v3/users/2caf5922d8514d3ab14f4d15e72ce625` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.193550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2caf5922d8514d3ab14f4d15e72ce625)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:42.195693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:42.256357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7342b9-3848-4544-a1d0-ff0c7ca023f0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:42.263144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:42.265554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.265901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.266245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.268420 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:42.270538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:42.295410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2caf5922d8514d3ab14f4d15e72ce625'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.305541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 956/1939] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:42 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:42.306690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.307270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2caf5922d8514d3ab14f4d15e72ce625'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.308179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2caf5922d8514d3ab14f4d15e72ce625'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.309279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:42.324697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaba371-6826-4e8b-8bf5-51dd6acdaaee admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:42.328293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2caf5922d8514d3ab14f4d15e72ce625'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.329315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2caf5922d8514d3ab14f4d15e72ce625'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.347554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 984/1940] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:42 2022] DELETE /identity/v3/users/2caf5922d8514d3ab14f4d15e72ce625 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:42.353544 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:42.354219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.354504 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.354804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] PATH_INFO: `/v3/users/2f123090eba448aca2d02c2111f80579` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.363260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f123090eba448aca2d02c2111f80579)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:42.364080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.365247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:42.366683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:42.433218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f123090eba448aca2d02c2111f80579'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.450425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f123090eba448aca2d02c2111f80579'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.451264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f123090eba448aca2d02c2111f80579'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.472891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f123090eba448aca2d02c2111f80579'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.473724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f123090eba448aca2d02c2111f80579'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.494488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 957/1941] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:42 2022] DELETE /identity/v3/users/2f123090eba448aca2d02c2111f80579 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:42.495449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1b4e8-8be7-426d-b8fa-d488a9396f8e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:42.495759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.497643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:42.518057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:42.550116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:42.627262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca7e1a-0e0f-474b-894e-2eec53614438 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:42.639593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3be8a-acfd-4a7f-9322-05c55ab6bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:42.640229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3be8a-acfd-4a7f-9322-05c55ab6bd7f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.640986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3be8a-acfd-4a7f-9322-05c55ab6bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.642588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3be8a-acfd-4a7f-9322-05c55ab6bd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.643093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:42.643739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.644009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.644272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] PATH_INFO: `/v3/projects/f7db5c3cf55e463f9b0b370839c96b90` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.645126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3be8a-acfd-4a7f-9322-05c55ab6bd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:42.646772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3be8a-acfd-4a7f-9322-05c55ab6bd7f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:42.649594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7db5c3cf55e463f9b0b370839c96b90)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:42.652057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:42.674379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 958/1942] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:42.675260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3be8a-acfd-4a7f-9322-05c55ab6bd7f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.696478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3be8a-acfd-4a7f-9322-05c55ab6bd7f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:42.711958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4090cdd-0772-4bdc-9300-616cf59df9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:42.711958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4090cdd-0772-4bdc-9300-616cf59df9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.711958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4090cdd-0772-4bdc-9300-616cf59df9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.711958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4090cdd-0772-4bdc-9300-616cf59df9cd admin admin] PATH_INFO: `/v3/projects/effbfe24efda429d9a90343a2b71c48e/users/594d581a1a40489d94c1393965a6f976/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.763386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4090cdd-0772-4bdc-9300-616cf59df9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=effbfe24efda429d9a90343a2b71c48e, user_id=594d581a1a40489d94c1393965a6f976, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:42.768374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4090cdd-0772-4bdc-9300-616cf59df9cd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:42.770368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7db5c3cf55e463f9b0b370839c96b90', 'request_id': 'req-de232a27-638b-4fd9-996d-d9cecfebdf05'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.789207 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7db5c3cf55e463f9b0b370839c96b90', 'request_id': 'req-de232a27-638b-4fd9-996d-d9cecfebdf05'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.801536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7db5c3cf55e463f9b0b370839c96b90', 'request_id': 'req-de232a27-638b-4fd9-996d-d9cecfebdf05'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.804061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 959/1943] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:42 2022] PUT /identity/v3/projects/effbfe24efda429d9a90343a2b71c48e/users/594d581a1a40489d94c1393965a6f976/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:42.808027 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4090cdd-0772-4bdc-9300-616cf59df9cd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.818802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] The token cache is being invalidate because project f7db5c3cf55e463f9b0b370839c96b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:42.819095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.819764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de232a27-638b-4fd9-996d-d9cecfebdf05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:42.825646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 985/1944] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:42 2022] DELETE /identity/v3/projects/f7db5c3cf55e463f9b0b370839c96b90 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:42.827244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f274c-4283-4ccb-8f0d-19e38239909a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.827527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f274c-4283-4ccb-8f0d-19e38239909a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.827804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f274c-4283-4ccb-8f0d-19e38239909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.853920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4c0f274c-4283-4ccb-8f0d-19e38239909a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:42.857066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4c0f274c-4283-4ccb-8f0d-19e38239909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:42.914189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4090cdd-0772-4bdc-9300-616cf59df9cd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:42.925817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:42.927026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.927415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.927756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.928724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:42.931985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:42.946850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 986/1945] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:42.948594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d56c-d197-45ec-baa8-71a0f4ea1f1d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.948868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d56c-d197-45ec-baa8-71a0f4ea1f1d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.949149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d56c-d197-45ec-baa8-71a0f4ea1f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.960172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 960/1946] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:42.961907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.964396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:42.979026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb5363-3c87-4855-9f3f-ee28068eb131 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:42.979259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-55a9d56c-d197-45ec-baa8-71a0f4ea1f1d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:42.981009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-55a9d56c-d197-45ec-baa8-71a0f4ea1f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:42.985072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879fbbf-07ac-4eff-b5b1-ca65f9ce7f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:42.986012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879fbbf-07ac-4eff-b5b1-ca65f9ce7f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:42.986755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879fbbf-07ac-4eff-b5b1-ca65f9ce7f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:42.987801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879fbbf-07ac-4eff-b5b1-ca65f9ce7f20 admin admin] PATH_INFO: `/v3/projects/effbfe24efda429d9a90343a2b71c48e/users/594d581a1a40489d94c1393965a6f976/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:42.994001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 987/1947] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:42.994949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879fbbf-07ac-4eff-b5b1-ca65f9ce7f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=effbfe24efda429d9a90343a2b71c48e, user_id=594d581a1a40489d94c1393965a6f976, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:42.995601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d56c-d197-45ec-baa8-71a0f4ea1f1d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:42.998075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879fbbf-07ac-4eff-b5b1-ca65f9ce7f20 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.010803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 961/1948] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:42 2022] PUT /identity/v3/projects/effbfe24efda429d9a90343a2b71c48e/users/594d581a1a40489d94c1393965a6f976/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:43.015843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d56c-d197-45ec-baa8-71a0f4ea1f1d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.032040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510f244-4ba6-4f7d-acc0-2f76a8027e47 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.032323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510f244-4ba6-4f7d-acc0-2f76a8027e47 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.032980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510f244-4ba6-4f7d-acc0-2f76a8027e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.066377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8510f244-4ba6-4f7d-acc0-2f76a8027e47 None None] MFA Rules not processed for user `594d581a1a40489d94c1393965a6f976`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:43.067772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8510f244-4ba6-4f7d-acc0-2f76a8027e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.088932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:43.109765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.110687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.111007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.111276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.111931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:43.112227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.114847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.130832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 988/1949] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:35:43.132132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.151328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.152722 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 962/1950] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:43.153433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510f244-4ba6-4f7d-acc0-2f76a8027e47 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.171546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510f244-4ba6-4f7d-acc0-2f76a8027e47 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.177903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.178595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.178911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.179185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.179845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.182270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d55ff-d4f6-457d-852e-c8a07498f6dc admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.182422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.187981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.189210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.190068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:43.190947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.190947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] PATH_INFO: `/v3/projects/b81c4253e5f6406f93505e859266658e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.192354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81c4253e5f6406f93505e859266658e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.197444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.228130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 963/1951] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:43 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:43.229516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.232946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.269410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81c4253e5f6406f93505e859266658e', 'request_id': 'req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.280681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81c4253e5f6406f93505e859266658e', 'request_id': 'req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.289592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81c4253e5f6406f93505e859266658e', 'request_id': 'req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.299555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fb27f-090b-4bbd-bdfc-f78d025e72cb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.302760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] The token cache is being invalidate because project b81c4253e5f6406f93505e859266658e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:43.303119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.303845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.306406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 989/1952] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:43 2022] DELETE /identity/v3/projects/b81c4253e5f6406f93505e859266658e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:43.307683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.309103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.358845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.359975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.360615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.361054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.364636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.395063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95cf2-9b29-45dc-bb5f-9290edca5ce4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.401413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.402105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.402222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.402563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.403390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.406150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.436664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 990/1953] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:43 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:43.437452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.439492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.452758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.453841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741541-74bf-4446-a9e2-bcc481a2eaa4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.454668 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.458727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.459491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.459767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.460036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] PATH_INFO: `/v3/users/508d96e3b2d24a4596d7769ebbfb39f7` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.461316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] RBAC: Authorizing `identity:delete_user(user_id=508d96e3b2d24a4596d7769ebbfb39f7)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.463869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.468032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 964/1954] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:43.469132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.483354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b167-27bc-4026-9547-ee90266161d4 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.493222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.495064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.495453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.495731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.496565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.505903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.534124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 965/1955] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:43.535902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.542641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.568667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087997-e2b2-4649-9c2b-bf6f6abdad5e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.577034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:43.579495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508d96e3b2d24a4596d7769ebbfb39f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.587514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.588394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.588743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.589138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] PATH_INFO: `/v3/projects/ac87c2498ad54abe8eadd4539055086b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.590681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac87c2498ad54abe8eadd4539055086b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.593090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508d96e3b2d24a4596d7769ebbfb39f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.594176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508d96e3b2d24a4596d7769ebbfb39f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.595729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.613369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508d96e3b2d24a4596d7769ebbfb39f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.614325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508d96e3b2d24a4596d7769ebbfb39f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.634552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 991/1956] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:43 2022] DELETE /identity/v3/users/508d96e3b2d24a4596d7769ebbfb39f7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:43.635659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.638303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.694070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:43.711600 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac87c2498ad54abe8eadd4539055086b', 'request_id': 'req-d78119ab-49b9-433b-956c-828aa930f88a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.723242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac87c2498ad54abe8eadd4539055086b', 'request_id': 'req-d78119ab-49b9-433b-956c-828aa930f88a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.736724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b991a0-2c31-4ad7-92be-c7d22b7ec5df admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.738315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac87c2498ad54abe8eadd4539055086b', 'request_id': 'req-d78119ab-49b9-433b-956c-828aa930f88a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.765950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] The token cache is being invalidate because project ac87c2498ad54abe8eadd4539055086b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:43.766284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.767594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:43.774975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 966/1957] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:43 2022] DELETE /identity/v3/projects/ac87c2498ad54abe8eadd4539055086b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:43.774975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.776327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.835202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233cfd5-a169-4471-a347-2db3d31ca731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.835928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233cfd5-a169-4471-a347-2db3d31ca731 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.836294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233cfd5-a169-4471-a347-2db3d31ca731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.836632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233cfd5-a169-4471-a347-2db3d31ca731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.837965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233cfd5-a169-4471-a347-2db3d31ca731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.840221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233cfd5-a169-4471-a347-2db3d31ca731 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.855553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 992/1958] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:43.857233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25635-c01b-4b72-8983-6202d8000676 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.857585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25635-c01b-4b72-8983-6202d8000676 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.857918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d25635-c01b-4b72-8983-6202d8000676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.879722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d8d25635-c01b-4b72-8983-6202d8000676 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:43.881082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d8d25635-c01b-4b72-8983-6202d8000676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.889552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 993/1959] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:43.893268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78119ab-49b9-433b-956c-828aa930f88a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.897826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25635-c01b-4b72-8983-6202d8000676 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:43.898934 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56407f-8472-4e11-ac8f-9afc575eda0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.899854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56407f-8472-4e11-ac8f-9afc575eda0b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.899989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d8d25635-c01b-4b72-8983-6202d8000676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.900435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56407f-8472-4e11-ac8f-9afc575eda0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.900877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56407f-8472-4e11-ac8f-9afc575eda0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.903123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4f56407f-8472-4e11-ac8f-9afc575eda0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:43.914176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d25635-c01b-4b72-8983-6202d8000676 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:43.919979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee92e36-d3d7-4f20-9032-f356155a7a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:43.921045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92e36-d3d7-4f20-9032-f356155a7a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.921414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92e36-d3d7-4f20-9032-f356155a7a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.921772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92e36-d3d7-4f20-9032-f356155a7a36 admin admin] PATH_INFO: `/v3/projects/490fc51450fd451bb94f6feb563761b8/users/7493ae67f426497b8595e37a06ac1f02/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:43.938152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee92e36-d3d7-4f20-9032-f356155a7a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=490fc51450fd451bb94f6feb563761b8, user_id=7493ae67f426497b8595e37a06ac1f02, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:43.940946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee92e36-d3d7-4f20-9032-f356155a7a36 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:43.958115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 994/1960] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:43 2022] PUT /identity/v3/projects/490fc51450fd451bb94f6feb563761b8/users/7493ae67f426497b8595e37a06ac1f02/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:43.960211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fe9d3-4a1a-45db-990f-52e67baac3eb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:43.960495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fe9d3-4a1a-45db-990f-52e67baac3eb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:43.960887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fe9d3-4a1a-45db-990f-52e67baac3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.000149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-619fe9d3-4a1a-45db-990f-52e67baac3eb None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:44.002183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-619fe9d3-4a1a-45db-990f-52e67baac3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:44.041376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56407f-8472-4e11-ac8f-9afc575eda0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.043402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56407f-8472-4e11-ac8f-9afc575eda0b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.056190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 967/1961] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:44.057802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b75d3-f6f0-4818-8f4c-f315d5989aeb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.058130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b75d3-f6f0-4818-8f4c-f315d5989aeb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.058739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b75d3-f6f0-4818-8f4c-f315d5989aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.103861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-870b75d3-f6f0-4818-8f4c-f315d5989aeb None None] MFA Rules not processed for user `7493ae67f426497b8595e37a06ac1f02`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:44.108295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:44.114081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-870b75d3-f6f0-4818-8f4c-f315d5989aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:44.184152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 995/1962] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:44.188408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fe9d3-4a1a-45db-990f-52e67baac3eb None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.198999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fe9d3-4a1a-45db-990f-52e67baac3eb None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.204112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.204725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.205173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.205458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.206275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:44.206559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.208379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.226608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 996/1963] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:35:44.231713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.251111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4131c5-d731-4b4d-b03a-edc96cdb66f2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.264434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658833ca-0d7e-473b-be82-99bb52e5118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.265463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658833ca-0d7e-473b-be82-99bb52e5118b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.265744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658833ca-0d7e-473b-be82-99bb52e5118b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.266000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658833ca-0d7e-473b-be82-99bb52e5118b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.266643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658833ca-0d7e-473b-be82-99bb52e5118b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.268205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 968/1964] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:44.275555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658833ca-0d7e-473b-be82-99bb52e5118b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.325876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b75d3-f6f0-4818-8f4c-f315d5989aeb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.331213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 997/1965] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:44 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:44.332181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658833ca-0d7e-473b-be82-99bb52e5118b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.340415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:44.417736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b75d3-f6f0-4818-8f4c-f315d5989aeb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.454095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658833ca-0d7e-473b-be82-99bb52e5118b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.460010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.461742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.462707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.463219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.464199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.466535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.506855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 998/1966] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:44 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:44.508114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.512823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:44.527323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729f908-ffe5-425f-a7b5-d85cfa98394d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.528329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920d11f-f34f-4a6e-8ec6-bc10cfc0c1ee admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.529217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729f908-ffe5-425f-a7b5-d85cfa98394d service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.529500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729f908-ffe5-425f-a7b5-d85cfa98394d service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.529769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729f908-ffe5-425f-a7b5-d85cfa98394d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.533779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.534434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.534699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.535005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] PATH_INFO: `/v3/projects/a45f03451c974c049e1277e9fd743a6c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.536144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a45f03451c974c049e1277e9fd743a6c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.549184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.617796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729f908-ffe5-425f-a7b5-d85cfa98394d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.619607 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729f908-ffe5-425f-a7b5-d85cfa98394d service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.633953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 969/1967] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:35:44.634828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729f908-ffe5-425f-a7b5-d85cfa98394d service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.645433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729f908-ffe5-425f-a7b5-d85cfa98394d service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.673802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a45f03451c974c049e1277e9fd743a6c', 'request_id': 'req-491447b3-cbdf-451b-a496-b856de1cc990'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:44.689935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a45f03451c974c049e1277e9fd743a6c', 'request_id': 'req-491447b3-cbdf-451b-a496-b856de1cc990'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:44.699141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a45f03451c974c049e1277e9fd743a6c', 'request_id': 'req-491447b3-cbdf-451b-a496-b856de1cc990'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:44.716157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] The token cache is being invalidate because project a45f03451c974c049e1277e9fd743a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:44.717318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:44.719103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-491447b3-cbdf-451b-a496-b856de1cc990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:44.724185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 999/1968] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:44 2022] DELETE /identity/v3/projects/a45f03451c974c049e1277e9fd743a6c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:44.732411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f8cbe-a551-4c51-9d43-167dc0987d9d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.732720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f8cbe-a551-4c51-9d43-167dc0987d9d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.732963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f8cbe-a551-4c51-9d43-167dc0987d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.764163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.764952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.765503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.766866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.767640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.775349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.775510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-455f8cbe-a551-4c51-9d43-167dc0987d9d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:44.776749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-455f8cbe-a551-4c51-9d43-167dc0987d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:44.789976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 970/1969] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:44.793548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.797582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:44.805644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1000/1970] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:44.807448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d04ef0-3e3d-4d02-a499-71cdd6a53614 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.807748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d04ef0-3e3d-4d02-a499-71cdd6a53614 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.808025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d04ef0-3e3d-4d02-a499-71cdd6a53614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.823016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303a282-47b6-43cd-a268-2f4c24a80630 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.824981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:44.829367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541e7e-21c8-47a5-a18d-492cc3b94516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.830482 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541e7e-21c8-47a5-a18d-492cc3b94516 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.833599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541e7e-21c8-47a5-a18d-492cc3b94516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.833599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541e7e-21c8-47a5-a18d-492cc3b94516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.833599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541e7e-21c8-47a5-a18d-492cc3b94516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.844518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c8d04ef0-3e3d-4d02-a499-71cdd6a53614 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:44.849938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c8d04ef0-3e3d-4d02-a499-71cdd6a53614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:44.851112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541e7e-21c8-47a5-a18d-492cc3b94516 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.862407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1001/1971] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:44.863593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d04ef0-3e3d-4d02-a499-71cdd6a53614 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.866097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 971/1972] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:35:44.871444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541e7e-21c8-47a5-a18d-492cc3b94516 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.878213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d04ef0-3e3d-4d02-a499-71cdd6a53614 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.884485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.885481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.885924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.886299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.887264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:44.887709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.889882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.896405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:44.905611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541e7e-21c8-47a5-a18d-492cc3b94516 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.910782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1002/1973] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:35:44.913783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d64d-71ea-44dc-b84d-fea6c7a19b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.915099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d64d-71ea-44dc-b84d-fea6c7a19b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.915939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:44.916519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d64d-71ea-44dc-b84d-fea6c7a19b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.917001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d64d-71ea-44dc-b84d-fea6c7a19b25 admin admin] PATH_INFO: `/v3/projects/8aa77dc33f624bc7a07b3bea98eb63aa/users/ba94e2aea38641b39aec5db33eb4678a/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.946474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1d64d-71ea-44dc-b84d-fea6c7a19b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa77dc33f624bc7a07b3bea98eb63aa, user_id=ba94e2aea38641b39aec5db33eb4678a, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.950057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2624283-ff1e-44e6-882c-d5fabd2d4dcd admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:44.951838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1d64d-71ea-44dc-b84d-fea6c7a19b25 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.966446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37c765-fe84-46e6-86fd-1fbd97e99aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:44.967162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37c765-fe84-46e6-86fd-1fbd97e99aef admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.967513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37c765-fe84-46e6-86fd-1fbd97e99aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.967728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37c765-fe84-46e6-86fd-1fbd97e99aef admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:44.968406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37c765-fe84-46e6-86fd-1fbd97e99aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:44.977116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37c765-fe84-46e6-86fd-1fbd97e99aef admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:44.983425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 972/1974] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:44 2022] PUT /identity/v3/projects/8aa77dc33f624bc7a07b3bea98eb63aa/users/ba94e2aea38641b39aec5db33eb4678a/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:44.994505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f27d90-87b7-4b64-a069-214f413c9138 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:44.995058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f27d90-87b7-4b64-a069-214f413c9138 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:44.995669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f27d90-87b7-4b64-a069-214f413c9138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.042867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1003/1975] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:44 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:45.046412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37c765-fe84-46e6-86fd-1fbd97e99aef admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.054024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-66f27d90-87b7-4b64-a069-214f413c9138 None None] MFA Rules not processed for user `ba94e2aea38641b39aec5db33eb4678a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:45.064057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-66f27d90-87b7-4b64-a069-214f413c9138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:45.194251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 973/1976] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:45.198511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f27d90-87b7-4b64-a069-214f413c9138 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.204321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37c765-fe84-46e6-86fd-1fbd97e99aef admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.210471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e8063-c212-436a-b312-6a37e7b078a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.212454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e8063-c212-436a-b312-6a37e7b078a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.212454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e8063-c212-436a-b312-6a37e7b078a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.212652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e8063-c212-436a-b312-6a37e7b078a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.213235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e8063-c212-436a-b312-6a37e7b078a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.215819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e8063-c212-436a-b312-6a37e7b078a9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.228613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f27d90-87b7-4b64-a069-214f413c9138 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.234632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e7af0-dc09-4f5a-914e-3ce1386eeb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.235569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e7af0-dc09-4f5a-914e-3ce1386eeb30 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.235836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e7af0-dc09-4f5a-914e-3ce1386eeb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.236170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e7af0-dc09-4f5a-914e-3ce1386eeb30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.236894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e7af0-dc09-4f5a-914e-3ce1386eeb30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.239154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e7af0-dc09-4f5a-914e-3ce1386eeb30 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.258947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1004/1977] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:45 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:45.263446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e8063-c212-436a-b312-6a37e7b078a9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.287232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e8063-c212-436a-b312-6a37e7b078a9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.291250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 974/1978] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:45 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:45.295727 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e7af0-dc09-4f5a-914e-3ce1386eeb30 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.297437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c7056-36b0-4c4f-a269-5f33cfc6405f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.298065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c7056-36b0-4c4f-a269-5f33cfc6405f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.298338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c7056-36b0-4c4f-a269-5f33cfc6405f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.298618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c7056-36b0-4c4f-a269-5f33cfc6405f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.300390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c7056-36b0-4c4f-a269-5f33cfc6405f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.307987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c7056-36b0-4c4f-a269-5f33cfc6405f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.335994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1005/1979] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:45.339870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c7056-36b0-4c4f-a269-5f33cfc6405f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.361027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c7056-36b0-4c4f-a269-5f33cfc6405f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.431719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e7af0-dc09-4f5a-914e-3ce1386eeb30 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.445156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a934c-47cb-4ed1-84e8-22f55d52e437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.445877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a934c-47cb-4ed1-84e8-22f55d52e437 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.446156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a934c-47cb-4ed1-84e8-22f55d52e437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.446509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a934c-47cb-4ed1-84e8-22f55d52e437 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.447411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a934c-47cb-4ed1-84e8-22f55d52e437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.449337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a934c-47cb-4ed1-84e8-22f55d52e437 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.512623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 975/1980] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:45 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:45.513772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce6db1-e653-425d-8be2-e1328271e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.515688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce6db1-e653-425d-8be2-e1328271e9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.515977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce6db1-e653-425d-8be2-e1328271e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.516695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a934c-47cb-4ed1-84e8-22f55d52e437 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.517153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce6db1-e653-425d-8be2-e1328271e9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.518075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce6db1-e653-425d-8be2-e1328271e9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.520192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce6db1-e653-425d-8be2-e1328271e9c5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.541964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a934c-47cb-4ed1-84e8-22f55d52e437 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.556209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dfb8b-d1c5-4e4c-ad4d-92f8176434b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.556968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfb8b-d1c5-4e4c-ad4d-92f8176434b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.557179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfb8b-d1c5-4e4c-ad4d-92f8176434b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.557524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dfb8b-d1c5-4e4c-ad4d-92f8176434b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.558239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dfb8b-d1c5-4e4c-ad4d-92f8176434b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.558759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1006/1981] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:35:45.563321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce6db1-e653-425d-8be2-e1328271e9c5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.565482 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dfb8b-d1c5-4e4c-ad4d-92f8176434b2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.603673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce6db1-e653-425d-8be2-e1328271e9c5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.615538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 976/1982] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:35:45.617384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dfb8b-d1c5-4e4c-ad4d-92f8176434b2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.634156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b9b14-10d7-4b2b-8088-301e7419b8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.636130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9b14-10d7-4b2b-8088-301e7419b8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.637899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9b14-10d7-4b2b-8088-301e7419b8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.640177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9b14-10d7-4b2b-8088-301e7419b8dd admin admin] PATH_INFO: `/v3/projects/e279a8dd7da242a7a3d15723eb8126b0/users/5b5b6438b4bd49a5b26bbd9d52e6c8f2/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.648096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dfb8b-d1c5-4e4c-ad4d-92f8176434b2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.655956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e1816-c71c-47cf-b467-b1781512ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.657998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1816-c71c-47cf-b467-b1781512ccf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.658399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1816-c71c-47cf-b467-b1781512ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.658834 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1816-c71c-47cf-b467-b1781512ccf8 admin admin] PATH_INFO: `/v3/projects/83fde45700ff491f8cfa7d925e7b88c4/users/d0168cf64ffa49d4ade0dd60ff25cfec/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.692280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b9b14-10d7-4b2b-8088-301e7419b8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e279a8dd7da242a7a3d15723eb8126b0, user_id=5b5b6438b4bd49a5b26bbd9d52e6c8f2, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.693577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e1816-c71c-47cf-b467-b1781512ccf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fde45700ff491f8cfa7d925e7b88c4, user_id=d0168cf64ffa49d4ade0dd60ff25cfec, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.699492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b9b14-10d7-4b2b-8088-301e7419b8dd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.700648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e1816-c71c-47cf-b467-b1781512ccf8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.728019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1007/1983] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:45 2022] PUT /identity/v3/projects/e279a8dd7da242a7a3d15723eb8126b0/users/5b5b6438b4bd49a5b26bbd9d52e6c8f2/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:45.734664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 977/1984] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:45 2022] PUT /identity/v3/projects/83fde45700ff491f8cfa7d925e7b88c4/users/d0168cf64ffa49d4ade0dd60ff25cfec/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:45.738085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a739f-8b1a-41b6-824e-4426175f6cd6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.738373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a739f-8b1a-41b6-824e-4426175f6cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.738520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e1816-c71c-47cf-b467-b1781512ccf8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.738652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a739f-8b1a-41b6-824e-4426175f6cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.752852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:45.786538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a06a739f-8b1a-41b6-824e-4426175f6cd6 None None] MFA Rules not processed for user `5b5b6438b4bd49a5b26bbd9d52e6c8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:45.787809 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a06a739f-8b1a-41b6-824e-4426175f6cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:45.812773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:45.906244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e1816-c71c-47cf-b467-b1781512ccf8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:45.927501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:45.928194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:45.929181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:45.929722 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:45.930154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:45.935320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:45.958311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1008/1985] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:45.963787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 978/1986] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:45.970181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a739f-8b1a-41b6-824e-4426175f6cd6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.972253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:45.974508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:45.994178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a739f-8b1a-41b6-824e-4426175f6cd6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:46.001425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402ea6e-8095-42a6-960c-ca68ba383a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:46.002201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ea6e-8095-42a6-960c-ca68ba383a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:46.002393 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ea6e-8095-42a6-960c-ca68ba383a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:46.002667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ea6e-8095-42a6-960c-ca68ba383a20 admin admin] PATH_INFO: `/v3/projects/83fde45700ff491f8cfa7d925e7b88c4/users/d0168cf64ffa49d4ade0dd60ff25cfec/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:46.005829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402ea6e-8095-42a6-960c-ca68ba383a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fde45700ff491f8cfa7d925e7b88c4, user_id=d0168cf64ffa49d4ade0dd60ff25cfec, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:46.008553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402ea6e-8095-42a6-960c-ca68ba383a20 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:46.027359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1009/1987] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:45 2022] PUT /identity/v3/projects/83fde45700ff491f8cfa7d925e7b88c4/users/d0168cf64ffa49d4ade0dd60ff25cfec/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:46.038427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c655-253a-46fc-bf27-71d234e05cbc None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:46.038817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c655-253a-46fc-bf27-71d234e05cbc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:46.039673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c655-253a-46fc-bf27-71d234e05cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:46.074276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892efd2-1892-42c1-93f1-0f181d88f224 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:46.111848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fbb3c655-253a-46fc-bf27-71d234e05cbc None None] MFA Rules not processed for user `d0168cf64ffa49d4ade0dd60ff25cfec`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:46.113225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fbb3c655-253a-46fc-bf27-71d234e05cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:46.156475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:46.157415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:46.157889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:46.158506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:46.259467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:46.260943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:46.271694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1010/1988] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:46.284728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3c655-253a-46fc-bf27-71d234e05cbc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:46.286478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fbb3c655-253a-46fc-bf27-71d234e05cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:46.293811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 979/1989] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:46.413390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3c655-253a-46fc-bf27-71d234e05cbc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:46.422035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c7488-c822-49a8-8f8a-223d98598abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:46.422755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7488-c822-49a8-8f8a-223d98598abc service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:46.423052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7488-c822-49a8-8f8a-223d98598abc service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:46.423319 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7488-c822-49a8-8f8a-223d98598abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:46.463232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:46.537711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c7488-c822-49a8-8f8a-223d98598abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:46.539210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c7488-c822-49a8-8f8a-223d98598abc service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:46.556558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1011/1990] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:49.251208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:49.253179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:49.269073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:49.346632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ba665-60a0-4f9d-bc6a-c7002d7c706d service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:49.352908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:49.353643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:49.354091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:49.354498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] PATH_INFO: `/v3/users/7493ae67f426497b8595e37a06ac1f02` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:49.355770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7493ae67f426497b8595e37a06ac1f02)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:49.357750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:49.414898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7493ae67f426497b8595e37a06ac1f02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:49.425389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7493ae67f426497b8595e37a06ac1f02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:49.426073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7493ae67f426497b8595e37a06ac1f02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:49.440150 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7493ae67f426497b8595e37a06ac1f02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:49.440894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7493ae67f426497b8595e37a06ac1f02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:49.453593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 980/1991] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:49 2022] DELETE /identity/v3/users/7493ae67f426497b8595e37a06ac1f02 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:49.458265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c169f9-534f-4399-8506-dd79d72e56d1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:49.458995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c169f9-534f-4399-8506-dd79d72e56d1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:49.458995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c169f9-534f-4399-8506-dd79d72e56d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:49.479436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e6c169f9-534f-4399-8506-dd79d72e56d1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:49.480612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e6c169f9-534f-4399-8506-dd79d72e56d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:49.540908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1012/1992] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:49.552053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:49.553478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:49.625160 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f2906-5241-49d7-8181-41ce79dd4222 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:49.631328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efffce-1a43-4b39-a677-07a32b85008d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:49.631973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efffce-1a43-4b39-a677-07a32b85008d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:49.632231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efffce-1a43-4b39-a677-07a32b85008d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:49.632498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efffce-1a43-4b39-a677-07a32b85008d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:49.642983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efffce-1a43-4b39-a677-07a32b85008d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:49.644568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efffce-1a43-4b39-a677-07a32b85008d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:49.656546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 981/1993] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:49.937637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c169f9-534f-4399-8506-dd79d72e56d1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:49.939166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e6c169f9-534f-4399-8506-dd79d72e56d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:49.957338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c169f9-534f-4399-8506-dd79d72e56d1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:49.964343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:49.965135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:49.965598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:49.966120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] PATH_INFO: `/v3/projects/490fc51450fd451bb94f6feb563761b8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:49.967397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490fc51450fd451bb94f6feb563761b8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:49.969354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.030659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490fc51450fd451bb94f6feb563761b8', 'request_id': 'req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:50.039665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490fc51450fd451bb94f6feb563761b8', 'request_id': 'req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:50.046979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490fc51450fd451bb94f6feb563761b8', 'request_id': 'req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:50.059502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] The token cache is being invalidate because project 490fc51450fd451bb94f6feb563761b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:50.059820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:50.060530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-911b9a76-2d3e-4df8-b64e-76a4ee7b54e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:50.062215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1013/1994] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:49 2022] DELETE /identity/v3/projects/490fc51450fd451bb94f6feb563761b8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:50.067491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17887b-2af4-49de-a6b5-e9ba8c761611 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.068343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17887b-2af4-49de-a6b5-e9ba8c761611 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.068343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17887b-2af4-49de-a6b5-e9ba8c761611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.093332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4a17887b-2af4-49de-a6b5-e9ba8c761611 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:50.094536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a17887b-2af4-49de-a6b5-e9ba8c761611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:50.123689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:50.162750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 982/1995] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:50.170773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced84662-01dc-47c8-90d8-9a78fbdb54f9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.171061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced84662-01dc-47c8-90d8-9a78fbdb54f9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.171339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced84662-01dc-47c8-90d8-9a78fbdb54f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.198928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ced84662-01dc-47c8-90d8-9a78fbdb54f9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:50.198928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ced84662-01dc-47c8-90d8-9a78fbdb54f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:50.201993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1014/1996] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:35:50.206400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17887b-2af4-49de-a6b5-e9ba8c761611 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.216478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17887b-2af4-49de-a6b5-e9ba8c761611 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.221640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.222258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.222536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.222843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.223492 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:35:50.224378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.225974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.243013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 983/1997] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:35:50.244319 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced84662-01dc-47c8-90d8-9a78fbdb54f9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.253882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced84662-01dc-47c8-90d8-9a78fbdb54f9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.258640 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8e63e-05b4-42cd-992f-95a86dbe7fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.259256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8e63e-05b4-42cd-992f-95a86dbe7fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.259896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8e63e-05b4-42cd-992f-95a86dbe7fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.259896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8e63e-05b4-42cd-992f-95a86dbe7fe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.260448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8e63e-05b4-42cd-992f-95a86dbe7fe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.262423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8e63e-05b4-42cd-992f-95a86dbe7fe2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.294985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1015/1998] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:50 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:50.299538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.311129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82894d01-7f28-43b8-b799-e94b1725351b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.386924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b4f37-04b0-40a5-a418-5f4eecc478ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.387562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b4f37-04b0-40a5-a418-5f4eecc478ef admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.387840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b4f37-04b0-40a5-a418-5f4eecc478ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.388107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b4f37-04b0-40a5-a418-5f4eecc478ef admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.388956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b4f37-04b0-40a5-a418-5f4eecc478ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.391043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b4f37-04b0-40a5-a418-5f4eecc478ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.417494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 984/1999] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:50 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:50.421669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8e63e-05b4-42cd-992f-95a86dbe7fe2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.432577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8e63e-05b4-42cd-992f-95a86dbe7fe2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.438607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43fdd-852f-4992-8dfa-1e654f53d693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.439293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43fdd-852f-4992-8dfa-1e654f53d693 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.439571 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43fdd-852f-4992-8dfa-1e654f53d693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.439834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43fdd-852f-4992-8dfa-1e654f53d693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.440493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f43fdd-852f-4992-8dfa-1e654f53d693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.442135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f43fdd-852f-4992-8dfa-1e654f53d693 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.453949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1016/2000] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:50.458081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b4f37-04b0-40a5-a418-5f4eecc478ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.470210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b4f37-04b0-40a5-a418-5f4eecc478ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.478862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003320cb-8167-4524-91de-aa433f04a702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.479664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003320cb-8167-4524-91de-aa433f04a702 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.479899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003320cb-8167-4524-91de-aa433f04a702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.480145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003320cb-8167-4524-91de-aa433f04a702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.480776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003320cb-8167-4524-91de-aa433f04a702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.482409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003320cb-8167-4524-91de-aa433f04a702 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.503900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43fdd-852f-4992-8dfa-1e654f53d693 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.510648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 985/2001] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:50.514460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43fdd-852f-4992-8dfa-1e654f53d693 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.522043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e351a-759c-4efd-b487-96c76a6d845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.522834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e351a-759c-4efd-b487-96c76a6d845e admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.523395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e351a-759c-4efd-b487-96c76a6d845e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.523670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e351a-759c-4efd-b487-96c76a6d845e admin admin] PATH_INFO: `/v3/projects/bb25a7eee31d4a47b99a89e640c0d5a6/users/497a9fddfd3d4cc097864ca08e68b63a/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.538603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:50.542669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e351a-759c-4efd-b487-96c76a6d845e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb25a7eee31d4a47b99a89e640c0d5a6, user_id=497a9fddfd3d4cc097864ca08e68b63a, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.550223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e351a-759c-4efd-b487-96c76a6d845e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.576890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1017/2002] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:50 2022] PUT /identity/v3/projects/bb25a7eee31d4a47b99a89e640c0d5a6/users/497a9fddfd3d4cc097864ca08e68b63a/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:50.578592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048656d-e9a5-4c8a-ae88-76462a337e22 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.578918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048656d-e9a5-4c8a-ae88-76462a337e22 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.579194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048656d-e9a5-4c8a-ae88-76462a337e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.612713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c048656d-e9a5-4c8a-ae88-76462a337e22 None None] MFA Rules not processed for user `497a9fddfd3d4cc097864ca08e68b63a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:50.614211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c048656d-e9a5-4c8a-ae88-76462a337e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:50.705108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 986/2003] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:50.709656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e351a-759c-4efd-b487-96c76a6d845e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.780981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e351a-759c-4efd-b487-96c76a6d845e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.786027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77bb89-f611-4be6-b8bb-c1cec375fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.786651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77bb89-f611-4be6-b8bb-c1cec375fc6f admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.786958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77bb89-f611-4be6-b8bb-c1cec375fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.787232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77bb89-f611-4be6-b8bb-c1cec375fc6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.787902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77bb89-f611-4be6-b8bb-c1cec375fc6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.789942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77bb89-f611-4be6-b8bb-c1cec375fc6f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.817822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1018/2004] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:50 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:50.821973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048656d-e9a5-4c8a-ae88-76462a337e22 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.832941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048656d-e9a5-4c8a-ae88-76462a337e22 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.859562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:50.892516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe7ed-1183-433b-a70e-8a83c948d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.893133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe7ed-1183-433b-a70e-8a83c948d9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.893408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe7ed-1183-433b-a70e-8a83c948d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.893670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe7ed-1183-433b-a70e-8a83c948d9c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.894290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe7ed-1183-433b-a70e-8a83c948d9c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.896273 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe7ed-1183-433b-a70e-8a83c948d9c7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.919549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 987/2005] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:50 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:35:50.924955 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77bb89-f611-4be6-b8bb-c1cec375fc6f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.936065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77bb89-f611-4be6-b8bb-c1cec375fc6f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.942011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75767c5c-424c-4f5e-bb6c-a5fcc2150287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.942646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75767c5c-424c-4f5e-bb6c-a5fcc2150287 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.942950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75767c5c-424c-4f5e-bb6c-a5fcc2150287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.943217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75767c5c-424c-4f5e-bb6c-a5fcc2150287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.943865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75767c5c-424c-4f5e-bb6c-a5fcc2150287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.945487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75767c5c-424c-4f5e-bb6c-a5fcc2150287 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.956398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1019/2006] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:35:50.960618 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe7ed-1183-433b-a70e-8a83c948d9c7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:50.971977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe7ed-1183-433b-a70e-8a83c948d9c7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:50.976724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d0fd2-a650-417d-badb-480b0a8425d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:50.977338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d0fd2-a650-417d-badb-480b0a8425d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:50.977612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d0fd2-a650-417d-badb-480b0a8425d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:50.977880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d0fd2-a650-417d-badb-480b0a8425d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:50.978570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d0fd2-a650-417d-badb-480b0a8425d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:50.980204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d0fd2-a650-417d-badb-480b0a8425d3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:50.993050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 988/2007] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:50.995228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75767c5c-424c-4f5e-bb6c-a5fcc2150287 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.005565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75767c5c-424c-4f5e-bb6c-a5fcc2150287 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.010442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed78863-4f20-48d3-9478-0f9725c382e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.011192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed78863-4f20-48d3-9478-0f9725c382e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.011475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed78863-4f20-48d3-9478-0f9725c382e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.011745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed78863-4f20-48d3-9478-0f9725c382e5 admin admin] PATH_INFO: `/v3/projects/f361cf57cca84361b3cd97b232568a36/users/4b396f55454e4ec99f8f012c77258b96/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.027743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed78863-4f20-48d3-9478-0f9725c382e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f361cf57cca84361b3cd97b232568a36, user_id=4b396f55454e4ec99f8f012c77258b96, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.030428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed78863-4f20-48d3-9478-0f9725c382e5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:51.045081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1020/2008] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:50 2022] PUT /identity/v3/projects/f361cf57cca84361b3cd97b232568a36/users/4b396f55454e4ec99f8f012c77258b96/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:51.057015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e58694-00af-490f-9d43-1ee77263c947 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.057543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e58694-00af-490f-9d43-1ee77263c947 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.057817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e58694-00af-490f-9d43-1ee77263c947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.090178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-58e58694-00af-490f-9d43-1ee77263c947 None None] MFA Rules not processed for user `4b396f55454e4ec99f8f012c77258b96`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:51.091427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-58e58694-00af-490f-9d43-1ee77263c947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:51.158809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 989/2009] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:51.163524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed78863-4f20-48d3-9478-0f9725c382e5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.232016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed78863-4f20-48d3-9478-0f9725c382e5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.237291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bfc65-90b2-485a-b784-16547326bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.237911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bfc65-90b2-485a-b784-16547326bd6a admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.238194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bfc65-90b2-485a-b784-16547326bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.238461 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bfc65-90b2-485a-b784-16547326bd6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.239148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bfc65-90b2-485a-b784-16547326bd6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.241052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bfc65-90b2-485a-b784-16547326bd6a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:51.270000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1021/2010] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:35:51 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:51.274409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e58694-00af-490f-9d43-1ee77263c947 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.285790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e58694-00af-490f-9d43-1ee77263c947 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.358387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62349576-1a1d-4ad4-9060-df9d3dbec435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.359125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349576-1a1d-4ad4-9060-df9d3dbec435 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.359312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349576-1a1d-4ad4-9060-df9d3dbec435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.359585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349576-1a1d-4ad4-9060-df9d3dbec435 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.360227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62349576-1a1d-4ad4-9060-df9d3dbec435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.362198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62349576-1a1d-4ad4-9060-df9d3dbec435 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:51.387020 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 990/2011] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:35:51 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:35:51.391355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bfc65-90b2-485a-b784-16547326bd6a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.402750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bfc65-90b2-485a-b784-16547326bd6a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.408919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9a549-05b2-4eb9-a991-11aca51cccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.409667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9a549-05b2-4eb9-a991-11aca51cccb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.410052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9a549-05b2-4eb9-a991-11aca51cccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.410289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9a549-05b2-4eb9-a991-11aca51cccb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.411158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9a549-05b2-4eb9-a991-11aca51cccb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.413079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9a549-05b2-4eb9-a991-11aca51cccb7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:51.427627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1022/2012] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:51.432763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62349576-1a1d-4ad4-9060-df9d3dbec435 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.443178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62349576-1a1d-4ad4-9060-df9d3dbec435 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.448322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb52275-da04-4cd8-9330-21956f9f4dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.448985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb52275-da04-4cd8-9330-21956f9f4dce admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.449254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb52275-da04-4cd8-9330-21956f9f4dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.449528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb52275-da04-4cd8-9330-21956f9f4dce admin admin] PATH_INFO: `/v3/projects/ca1aac15ae794f9b9c375feb9d1513d5/users/5af2eb45184d484faa24efb2452e0e03/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.466969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb52275-da04-4cd8-9330-21956f9f4dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1aac15ae794f9b9c375feb9d1513d5, user_id=5af2eb45184d484faa24efb2452e0e03, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.469741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb52275-da04-4cd8-9330-21956f9f4dce admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:51.481899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:51.486934 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 991/2013] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:51 2022] PUT /identity/v3/projects/ca1aac15ae794f9b9c375feb9d1513d5/users/5af2eb45184d484faa24efb2452e0e03/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:51.491578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9a549-05b2-4eb9-a991-11aca51cccb7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.502537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9a549-05b2-4eb9-a991-11aca51cccb7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.514990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:51.564507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d50a1-c4a0-4251-95a9-f6bfc805a161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.565264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d50a1-c4a0-4251-95a9-f6bfc805a161 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.565542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d50a1-c4a0-4251-95a9-f6bfc805a161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.565809 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d50a1-c4a0-4251-95a9-f6bfc805a161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.566507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d50a1-c4a0-4251-95a9-f6bfc805a161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.568219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d50a1-c4a0-4251-95a9-f6bfc805a161 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:51.582163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1023/2014] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:35:51.583813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb52275-da04-4cd8-9330-21956f9f4dce admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.595168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb52275-da04-4cd8-9330-21956f9f4dce admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.603323 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.604312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.604416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.604648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] PATH_INFO: `/v3/projects/ca1aac15ae794f9b9c375feb9d1513d5/users/5af2eb45184d484faa24efb2452e0e03/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.606565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1aac15ae794f9b9c375feb9d1513d5, user_id=5af2eb45184d484faa24efb2452e0e03, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.609281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:51.629496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 992/2015] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:35:51 2022] PUT /identity/v3/projects/ca1aac15ae794f9b9c375feb9d1513d5/users/5af2eb45184d484faa24efb2452e0e03/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:51.632009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d276-7d65-49f7-bc52-163066beaa36 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.632354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d276-7d65-49f7-bc52-163066beaa36 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.632683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d276-7d65-49f7-bc52-163066beaa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.662396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-49d6d276-7d65-49f7-bc52-163066beaa36 None None] MFA Rules not processed for user `5af2eb45184d484faa24efb2452e0e03`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:51.663696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-49d6d276-7d65-49f7-bc52-163066beaa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:51.667838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.669820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:51.748539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9bea2-7d76-4648-adb0-71dca365e6ac admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.752190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.752820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.753099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.753369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.755252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:51.762790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1024/2016] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:51.830316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.832029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:51.843486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 993/2017] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:51.884726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6d276-7d65-49f7-bc52-163066beaa36 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:51.886140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-49d6d276-7d65-49f7-bc52-163066beaa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:51.899757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6d276-7d65-49f7-bc52-163066beaa36 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:51.903470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac61677-8d10-4087-94ff-be7b15412302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:51.904082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac61677-8d10-4087-94ff-be7b15412302 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:51.904355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac61677-8d10-4087-94ff-be7b15412302 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:51.904620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac61677-8d10-4087-94ff-be7b15412302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:51.986150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac61677-8d10-4087-94ff-be7b15412302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:51.988216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac61677-8d10-4087-94ff-be7b15412302 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:52.003768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1025/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:53.615240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:53.624769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b00d81-5628-439a-9f6c-9cfe6fedf71a service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:53.628901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:53.629511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:53.629787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:53.630054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:53.688013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:53.689431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:53.698507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 994/2019] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:54.378263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac61677-8d10-4087-94ff-be7b15412302 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:54.389160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac61677-8d10-4087-94ff-be7b15412302 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:54.393200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa21bc-7fc8-4491-89ae-3cf67377d6ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:54.393871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa21bc-7fc8-4491-89ae-3cf67377d6ee service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:54.394145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa21bc-7fc8-4491-89ae-3cf67377d6ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:54.394414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa21bc-7fc8-4491-89ae-3cf67377d6ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:54.410829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa21bc-7fc8-4491-89ae-3cf67377d6ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:54.410829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa21bc-7fc8-4491-89ae-3cf67377d6ee service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:54.418639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1026/2020] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:58.011670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:58.013625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:58.085161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:58.116440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fee7c5-50ba-45d8-bae6-a90ca173ba8b service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:58.121009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:58.121678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:58.122758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:58.123210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] PATH_INFO: `/v3/users/ba94e2aea38641b39aec5db33eb4678a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:58.124610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba94e2aea38641b39aec5db33eb4678a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:58.126906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:58.188008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba94e2aea38641b39aec5db33eb4678a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.196787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba94e2aea38641b39aec5db33eb4678a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.197390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba94e2aea38641b39aec5db33eb4678a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.209572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba94e2aea38641b39aec5db33eb4678a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.210128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba94e2aea38641b39aec5db33eb4678a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.223925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 995/2021] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:58 2022] DELETE /identity/v3/users/ba94e2aea38641b39aec5db33eb4678a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:58.228473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc187-1fb5-4fb9-b917-a55b35fbd770 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:58.228925 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc187-1fb5-4fb9-b917-a55b35fbd770 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:58.229052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc187-1fb5-4fb9-b917-a55b35fbd770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:58.249129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-60dfc187-1fb5-4fb9-b917-a55b35fbd770 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:58.250844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-60dfc187-1fb5-4fb9-b917-a55b35fbd770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:58.268445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:58.311223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1027/2022] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:58.321264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:58.322743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:58.384251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecbe3e-8cb4-43ba-9cb8-ad33379ab456 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:58.388353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dced5-13a1-402a-a182-97884d50464b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:58.389036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dced5-13a1-402a-a182-97884d50464b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:58.389230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dced5-13a1-402a-a182-97884d50464b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:58.389490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dced5-13a1-402a-a182-97884d50464b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:58.400391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dced5-13a1-402a-a182-97884d50464b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:58.401933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dced5-13a1-402a-a182-97884d50464b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:58.411704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 996/2023] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:35:58.412140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc187-1fb5-4fb9-b917-a55b35fbd770 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:58.413802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-60dfc187-1fb5-4fb9-b917-a55b35fbd770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:58.426431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc187-1fb5-4fb9-b917-a55b35fbd770 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:58.431268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:58.431967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:58.432241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:58.432584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] PATH_INFO: `/v3/users/497a9fddfd3d4cc097864ca08e68b63a` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:58.434056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=497a9fddfd3d4cc097864ca08e68b63a)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:58.436465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:58.491844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497a9fddfd3d4cc097864ca08e68b63a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.499980 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497a9fddfd3d4cc097864ca08e68b63a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.500761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497a9fddfd3d4cc097864ca08e68b63a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.514651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497a9fddfd3d4cc097864ca08e68b63a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.515740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497a9fddfd3d4cc097864ca08e68b63a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.531390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1028/2024] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:58 2022] DELETE /identity/v3/users/497a9fddfd3d4cc097864ca08e68b63a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:35:58.536079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcecb7e-5d4d-4ed4-9a41-07ff61d6f8f1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:58.536361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcecb7e-5d4d-4ed4-9a41-07ff61d6f8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:58.536614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcecb7e-5d4d-4ed4-9a41-07ff61d6f8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:58.575079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0dcecb7e-5d4d-4ed4-9a41-07ff61d6f8f1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:35:58.576369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0dcecb7e-5d4d-4ed4-9a41-07ff61d6f8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:58.632054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:58.644897 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 997/2025] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:35:58.655101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:58.656629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:58.703701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcecb7e-5d4d-4ed4-9a41-07ff61d6f8f1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:58.705888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0dcecb7e-5d4d-4ed4-9a41-07ff61d6f8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:58.720093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcecb7e-5d4d-4ed4-9a41-07ff61d6f8f1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:58.727979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7cd4c-060c-4073-bf6d-a6c9ebf65a2d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:58.729733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:58.730431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:58.730731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:58.731009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] PATH_INFO: `/v3/projects/8aa77dc33f624bc7a07b3bea98eb63aa` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:58.732415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aa77dc33f624bc7a07b3bea98eb63aa)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:58.734629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:58.734799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f8ea6-f15c-4915-aea1-f54566d3f06b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:58.735448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8ea6-f15c-4915-aea1-f54566d3f06b service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:58.735671 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8ea6-f15c-4915-aea1-f54566d3f06b service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:58.735941 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8ea6-f15c-4915-aea1-f54566d3f06b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:58.746575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f8ea6-f15c-4915-aea1-f54566d3f06b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:58.748178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f8ea6-f15c-4915-aea1-f54566d3f06b service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:58.763186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1029/2026] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:35:58.804172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aa77dc33f624bc7a07b3bea98eb63aa', 'request_id': 'req-44759576-e6ac-4fdb-8d79-cfdad7505e36'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.811560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aa77dc33f624bc7a07b3bea98eb63aa', 'request_id': 'req-44759576-e6ac-4fdb-8d79-cfdad7505e36'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.821415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aa77dc33f624bc7a07b3bea98eb63aa', 'request_id': 'req-44759576-e6ac-4fdb-8d79-cfdad7505e36'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.837380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] The token cache is being invalidate because project 8aa77dc33f624bc7a07b3bea98eb63aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:58.837670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.838409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:58.840223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 998/2027] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:58 2022] DELETE /identity/v3/projects/8aa77dc33f624bc7a07b3bea98eb63aa => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:58.844433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f8ea6-f15c-4915-aea1-f54566d3f06b service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:58.866570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f8ea6-f15c-4915-aea1-f54566d3f06b service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:58.913078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:58.945114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:58.945867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:58.946293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:58.946582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] PATH_INFO: `/v3/users/d0168cf64ffa49d4ade0dd60ff25cfec` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:58.947843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0168cf64ffa49d4ade0dd60ff25cfec)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:58.949978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:59.010609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0168cf64ffa49d4ade0dd60ff25cfec'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.020001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0168cf64ffa49d4ade0dd60ff25cfec'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.020740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0168cf64ffa49d4ade0dd60ff25cfec'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.036340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0168cf64ffa49d4ade0dd60ff25cfec'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.036960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0168cf64ffa49d4ade0dd60ff25cfec'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.050877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1030/2028] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:58 2022] DELETE /identity/v3/users/d0168cf64ffa49d4ade0dd60ff25cfec => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:59.125451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:59.126941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:59.147283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44759576-e6ac-4fdb-8d79-cfdad7505e36 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:59.224015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:59.224745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:59.225089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:59.225423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] PATH_INFO: `/v3/projects/bb25a7eee31d4a47b99a89e640c0d5a6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:59.226963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb25a7eee31d4a47b99a89e640c0d5a6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:59.229106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:59.340353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb25a7eee31d4a47b99a89e640c0d5a6', 'request_id': 'req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.340353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb25a7eee31d4a47b99a89e640c0d5a6', 'request_id': 'req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.349326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb25a7eee31d4a47b99a89e640c0d5a6', 'request_id': 'req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.365565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] The token cache is being invalidate because project bb25a7eee31d4a47b99a89e640c0d5a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:35:59.365942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.366911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.368957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 999/2029] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:59 2022] DELETE /identity/v3/projects/bb25a7eee31d4a47b99a89e640c0d5a6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:59.375354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:59.376810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:59.402559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec8533-b0df-4300-8a62-b7920c6d90f4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:59.541146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:59.542072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:59.542530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:59.542964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] PATH_INFO: `/v3/users/4b396f55454e4ec99f8f012c77258b96` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:59.546217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b396f55454e4ec99f8f012c77258b96)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:59.548438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:59.631330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b396f55454e4ec99f8f012c77258b96'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.640837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b396f55454e4ec99f8f012c77258b96'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.644619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b396f55454e4ec99f8f012c77258b96'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.656588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:59.673846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b396f55454e4ec99f8f012c77258b96'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.675045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2a64b230-8052-48f6-8ef8-3c9a64c4f2ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b396f55454e4ec99f8f012c77258b96'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.689755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1031/2030] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:35:59 2022] DELETE /identity/v3/users/4b396f55454e4ec99f8f012c77258b96 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:35:59.785865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:35:59.787405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:35:59.808085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd1a08-1f67-4c9b-9bda-37040f68b23a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:35:59.827303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:35:59.911871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:35:59.912893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:35:59.913378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:35:59.913832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] PATH_INFO: `/v3/projects/83fde45700ff491f8cfa7d925e7b88c4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:35:59.920762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fde45700ff491f8cfa7d925e7b88c4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:35:59.923994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:35:59.989324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fde45700ff491f8cfa7d925e7b88c4', 'request_id': 'req-34769a8a-6e88-4214-8fd6-62556da06ad3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:35:59.999060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fde45700ff491f8cfa7d925e7b88c4', 'request_id': 'req-34769a8a-6e88-4214-8fd6-62556da06ad3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.009756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fde45700ff491f8cfa7d925e7b88c4', 'request_id': 'req-34769a8a-6e88-4214-8fd6-62556da06ad3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.022233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] The token cache is being invalidate because project 83fde45700ff491f8cfa7d925e7b88c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:00.022690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.023378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.025119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1000/2031] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:35:59 2022] DELETE /identity/v3/projects/83fde45700ff491f8cfa7d925e7b88c4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:00.032124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151eb4ff-7dc7-4bb8-a143-d1d7766da280 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.032353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151eb4ff-7dc7-4bb8-a143-d1d7766da280 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.032798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151eb4ff-7dc7-4bb8-a143-d1d7766da280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.061603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-151eb4ff-7dc7-4bb8-a143-d1d7766da280 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:00.063460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-151eb4ff-7dc7-4bb8-a143-d1d7766da280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:00.064346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.067859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:00.148231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1032/2032] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:00.157112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572722b-cca9-42de-ba31-81dac483c82b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.157620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572722b-cca9-42de-ba31-81dac483c82b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.157943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572722b-cca9-42de-ba31-81dac483c82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.163407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34769a8a-6e88-4214-8fd6-62556da06ad3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:00.168348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:00.169315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.169784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.170432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] PATH_INFO: `/v3/projects/f361cf57cca84361b3cd97b232568a36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.171786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f361cf57cca84361b3cd97b232568a36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:00.173710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:00.184278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9572722b-cca9-42de-ba31-81dac483c82b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:00.185807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9572722b-cca9-42de-ba31-81dac483c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:00.198901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1033/2033] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:00.200082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572722b-cca9-42de-ba31-81dac483c82b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.212527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572722b-cca9-42de-ba31-81dac483c82b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:00.223058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:00.224353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.225168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.225284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.226419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:00.227481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:00.229510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:00.236409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:00.241214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f361cf57cca84361b3cd97b232568a36', 'request_id': 'req-6af335cb-8d6b-4b80-b771-34b944f6471a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.247512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1034/2034] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:00.251893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.252056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f361cf57cca84361b3cd97b232568a36', 'request_id': 'req-6af335cb-8d6b-4b80-b771-34b944f6471a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.262662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f361cf57cca84361b3cd97b232568a36', 'request_id': 'req-6af335cb-8d6b-4b80-b771-34b944f6471a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.265873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e536ce-1f72-4830-9428-2805ee1346fb admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:00.275784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] The token cache is being invalidate because project f361cf57cca84361b3cd97b232568a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:00.276185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.277024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.279189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1001/2035] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:00 2022] DELETE /identity/v3/projects/f361cf57cca84361b3cd97b232568a36 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:00.284406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.286639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:00.331988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:00.332788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.333149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.333460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.334289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:00.336379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:00.371448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af335cb-8d6b-4b80-b771-34b944f6471a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:00.378418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:00.379386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.379845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.380051 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] PATH_INFO: `/v3/users/5af2eb45184d484faa24efb2452e0e03` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.381435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5af2eb45184d484faa24efb2452e0e03)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:00.383465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:00.395274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1035/2036] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:00 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:00.399062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.401433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:00.445075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5af2eb45184d484faa24efb2452e0e03'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.454044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5af2eb45184d484faa24efb2452e0e03'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.454868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5af2eb45184d484faa24efb2452e0e03'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.460289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:00.481029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5af2eb45184d484faa24efb2452e0e03'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.482015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5af2eb45184d484faa24efb2452e0e03'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:00.500377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb997f-fff4-4d5d-ad40-c599158e7606 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:00.502759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1002/2037] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:00 2022] DELETE /identity/v3/users/5af2eb45184d484faa24efb2452e0e03 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:00.566366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83266-a228-47b8-b60d-bdffab03251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:00.567025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83266-a228-47b8-b60d-bdffab03251e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.567301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83266-a228-47b8-b60d-bdffab03251e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.567578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83266-a228-47b8-b60d-bdffab03251e admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.568212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b83266-a228-47b8-b60d-bdffab03251e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:00.570138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b83266-a228-47b8-b60d-bdffab03251e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:00.594677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1036/2038] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:00 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:00.600911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.603731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:00.618411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6daa7f-7b54-427a-9bc4-f908854a8b50 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:00.628397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997359-a274-4324-a379-b29c05de2031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:00.629041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997359-a274-4324-a379-b29c05de2031 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.629333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997359-a274-4324-a379-b29c05de2031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.629598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7997359-a274-4324-a379-b29c05de2031 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.630268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7997359-a274-4324-a379-b29c05de2031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:00.631988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7997359-a274-4324-a379-b29c05de2031 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:00.645730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1003/2039] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:00.649862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83266-a228-47b8-b60d-bdffab03251e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.660325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83266-a228-47b8-b60d-bdffab03251e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:00.666045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e237e-ad12-480f-8bfe-5eb167effec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:00.666685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e237e-ad12-480f-8bfe-5eb167effec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.666994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e237e-ad12-480f-8bfe-5eb167effec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.667266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e237e-ad12-480f-8bfe-5eb167effec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.667929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e237e-ad12-480f-8bfe-5eb167effec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:00.669676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e237e-ad12-480f-8bfe-5eb167effec2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:00.682775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1037/2040] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:00.687289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997359-a274-4324-a379-b29c05de2031 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.698780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7997359-a274-4324-a379-b29c05de2031 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:00.707763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:00.708857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.709352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.709772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] PATH_INFO: `/v3/projects/2ce8b9ec039b4e24adfff9bed995a945/users/5068ee7196394acfb215418024d647a5/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.729023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce8b9ec039b4e24adfff9bed995a945, user_id=5068ee7196394acfb215418024d647a5, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:00.731967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:00.761665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1004/2041] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:00 2022] PUT /identity/v3/projects/2ce8b9ec039b4e24adfff9bed995a945/users/5068ee7196394acfb215418024d647a5/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:00.763635 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad368-9f68-43a8-b0fb-557ad74f8e56 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:00.763932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad368-9f68-43a8-b0fb-557ad74f8e56 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:00.764238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ad368-9f68-43a8-b0fb-557ad74f8e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:00.804862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1a3ad368-9f68-43a8-b0fb-557ad74f8e56 None None] MFA Rules not processed for user `5068ee7196394acfb215418024d647a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:00.808050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1a3ad368-9f68-43a8-b0fb-557ad74f8e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:00.905910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1038/2042] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:00.917980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:00.929107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:00.952491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:01.011498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58006c-d952-4a7f-96d1-aa9d978dc705 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:01.016582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:01.018252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.019179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.019555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.033361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:01.034841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:01.084162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1005/2043] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:01.120908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ad368-9f68-43a8-b0fb-557ad74f8e56 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:01.128022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1a3ad368-9f68-43a8-b0fb-557ad74f8e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:01.256149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:01.270783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ad368-9f68-43a8-b0fb-557ad74f8e56 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:01.278664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:01.279570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.279908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.280784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] PATH_INFO: `/v3/users/54edc7e36be14e3382c757c4e1d651fd` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.282231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54edc7e36be14e3382c757c4e1d651fd)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:01.286643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:01.317483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:01.319853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:01.341446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e727b9-b2f8-4aa4-978b-3347d773c865 service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:01.349569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:01.350250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.350525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.350953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] PATH_INFO: `/v3/projects/ca1aac15ae794f9b9c375feb9d1513d5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.353072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1aac15ae794f9b9c375feb9d1513d5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:01.356981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:01.384363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54edc7e36be14e3382c757c4e1d651fd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.396893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54edc7e36be14e3382c757c4e1d651fd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.399167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54edc7e36be14e3382c757c4e1d651fd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.427327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54edc7e36be14e3382c757c4e1d651fd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.428997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dc3d19d7-9601-4833-a7b6-31ef76bfb020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54edc7e36be14e3382c757c4e1d651fd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.453747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1039/2044] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:01 2022] DELETE /identity/v3/users/54edc7e36be14e3382c757c4e1d651fd => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:01.456096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef942d0c-a313-4260-8423-33706b920751 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.456389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef942d0c-a313-4260-8423-33706b920751 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.456663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef942d0c-a313-4260-8423-33706b920751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.461296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1aac15ae794f9b9c375feb9d1513d5', 'request_id': 'req-72ed6676-8b69-4c73-9b2c-fda9b629773f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.469039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1aac15ae794f9b9c375feb9d1513d5', 'request_id': 'req-72ed6676-8b69-4c73-9b2c-fda9b629773f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.479433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1aac15ae794f9b9c375feb9d1513d5', 'request_id': 'req-72ed6676-8b69-4c73-9b2c-fda9b629773f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.489786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ef942d0c-a313-4260-8423-33706b920751 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:01.491682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ef942d0c-a313-4260-8423-33706b920751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:01.498465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] The token cache is being invalidate because project ca1aac15ae794f9b9c375feb9d1513d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:01.498465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.499253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72ed6676-8b69-4c73-9b2c-fda9b629773f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:01.500942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1006/2045] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:01 2022] DELETE /identity/v3/projects/ca1aac15ae794f9b9c375feb9d1513d5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:01.512481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b70c0d-57d7-47cb-a50b-d9072b038fbc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.512481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b70c0d-57d7-47cb-a50b-d9072b038fbc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.512481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b70c0d-57d7-47cb-a50b-d9072b038fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.538959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-09b70c0d-57d7-47cb-a50b-d9072b038fbc None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:01.540702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-09b70c0d-57d7-47cb-a50b-d9072b038fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:01.584075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1040/2046] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:01.603979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef942d0c-a313-4260-8423-33706b920751 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:01.607607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ef942d0c-a313-4260-8423-33706b920751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:01.666997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1007/2047] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:01.675525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dedb29-048f-4ca5-af0b-4c9f67882580 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.676025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dedb29-048f-4ca5-af0b-4c9f67882580 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.676691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dedb29-048f-4ca5-af0b-4c9f67882580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.711072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef942d0c-a313-4260-8423-33706b920751 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:01.716154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2981862-e577-444b-bf13-1fa0f5b12689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:01.716896 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981862-e577-444b-bf13-1fa0f5b12689 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.717270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981862-e577-444b-bf13-1fa0f5b12689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.717636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981862-e577-444b-bf13-1fa0f5b12689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.729708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2981862-e577-444b-bf13-1fa0f5b12689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:01.731305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2981862-e577-444b-bf13-1fa0f5b12689 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:01.739635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b3dedb29-048f-4ca5-af0b-4c9f67882580 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:01.740753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b3dedb29-048f-4ca5-af0b-4c9f67882580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:01.755209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1041/2048] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:36:01.783908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1008/2049] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:01.791480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2981862-e577-444b-bf13-1fa0f5b12689 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:01.804414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2981862-e577-444b-bf13-1fa0f5b12689 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:01.824875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:01.825517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.825803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.826074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.826822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:01.827051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:01.828660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:01.848160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1042/2050] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:01.852672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dedb29-048f-4ca5-af0b-4c9f67882580 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:01.866169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dedb29-048f-4ca5-af0b-4c9f67882580 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:01.876118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5e332-8e3f-41c1-9876-164c0655121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:01.876809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e332-8e3f-41c1-9876-164c0655121b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:01.877094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e332-8e3f-41c1-9876-164c0655121b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:01.877365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e332-8e3f-41c1-9876-164c0655121b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:01.878082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5e332-8e3f-41c1-9876-164c0655121b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:01.880149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5e332-8e3f-41c1-9876-164c0655121b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:01.918959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1009/2051] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:01 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:01.923238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:01.935342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc32f7-cb84-4650-b2da-ae133d70635a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:01.940675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:02.005954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8dda0-0d3e-40ef-ab6c-8fcf086cf241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:02.006760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8dda0-0d3e-40ef-ab6c-8fcf086cf241 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:02.007086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8dda0-0d3e-40ef-ab6c-8fcf086cf241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:02.007361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8dda0-0d3e-40ef-ab6c-8fcf086cf241 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:02.008229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8dda0-0d3e-40ef-ab6c-8fcf086cf241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:02.010079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8dda0-0d3e-40ef-ab6c-8fcf086cf241 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:02.050904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1043/2052] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:01 2022] POST /identity/v3/users => generated 282 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:02.051749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5e332-8e3f-41c1-9876-164c0655121b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:02.100911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5e332-8e3f-41c1-9876-164c0655121b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:02.115489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f642d03-3283-4322-abda-4da9867ac0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:02.117774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f642d03-3283-4322-abda-4da9867ac0ac admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:02.118797 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f642d03-3283-4322-abda-4da9867ac0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:02.119102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f642d03-3283-4322-abda-4da9867ac0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:02.122834 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f642d03-3283-4322-abda-4da9867ac0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:02.127649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f642d03-3283-4322-abda-4da9867ac0ac admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:02.157632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1010/2053] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:02.162052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8dda0-0d3e-40ef-ab6c-8fcf086cf241 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:02.177164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8dda0-0d3e-40ef-ab6c-8fcf086cf241 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:02.182580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:02.183542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:02.183893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:02.184218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:02.185839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:02.187951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:02.207482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1044/2054] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:02.207819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f642d03-3283-4322-abda-4da9867ac0ac admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:02.215091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:02.230053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f642d03-3283-4322-abda-4da9867ac0ac admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:02.238040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59617a54-55ef-49f2-861d-f5427ee87ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:02.240139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617a54-55ef-49f2-861d-f5427ee87ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:02.240651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617a54-55ef-49f2-861d-f5427ee87ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:02.241199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617a54-55ef-49f2-861d-f5427ee87ccf admin admin] PATH_INFO: `/v3/projects/8cd369e63e034e6db409c5a7a1ad5260/users/6b46b894e2ec49e882572cd50ff6bd47/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:02.280465 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59617a54-55ef-49f2-861d-f5427ee87ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd369e63e034e6db409c5a7a1ad5260, user_id=6b46b894e2ec49e882572cd50ff6bd47, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:02.287331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59617a54-55ef-49f2-861d-f5427ee87ccf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:02.319580 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1011/2055] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:02 2022] PUT /identity/v3/projects/8cd369e63e034e6db409c5a7a1ad5260/users/6b46b894e2ec49e882572cd50ff6bd47/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:02.322508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:02.328770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:02.347864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3311d-0915-4841-b99e-af153d9e5426 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:02.348192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3311d-0915-4841-b99e-af153d9e5426 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:02.348484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3311d-0915-4841-b99e-af153d9e5426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:02.369384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55490-a9b1-4da2-8575-0520d91ca2f7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:02.410252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-aad3311d-0915-4841-b99e-af153d9e5426 None None] MFA Rules not processed for user `6b46b894e2ec49e882572cd50ff6bd47`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:02.411643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aad3311d-0915-4841-b99e-af153d9e5426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:02.483533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:02.485595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:02.486048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:02.486460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] PATH_INFO: `/v3/projects/99e46fc8e45b4ef685c2a95aca3b61c6` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:02.488792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e46fc8e45b4ef685c2a95aca3b61c6)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:02.491334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:02.515062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1012/2056] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:02.533913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3311d-0915-4841-b99e-af153d9e5426 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:02.539212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aad3311d-0915-4841-b99e-af153d9e5426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:02.603229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e46fc8e45b4ef685c2a95aca3b61c6', 'request_id': 'req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.622206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e46fc8e45b4ef685c2a95aca3b61c6', 'request_id': 'req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.634828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e46fc8e45b4ef685c2a95aca3b61c6', 'request_id': 'req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.649850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3311d-0915-4841-b99e-af153d9e5426 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:02.659631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] The token cache is being invalidate because project 99e46fc8e45b4ef685c2a95aca3b61c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:02.659949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.660748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.662670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1045/2057] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:02 2022] DELETE /identity/v3/projects/99e46fc8e45b4ef685c2a95aca3b61c6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:02.666825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:02.668251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:02.692156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:02.715598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:02.716283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:02.716566 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:02.716841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:02.719371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:02.750434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e89a9a-a2f3-4758-8f3d-c43599ff3a79 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:02.757465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:02.758352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:02.758639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:02.759007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] PATH_INFO: `/v3/users/594d581a1a40489d94c1393965a6f976` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:02.760218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=594d581a1a40489d94c1393965a6f976)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:02.762767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:02.772289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:02.802610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:02.805052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:02.816745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1013/2058] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:02.839483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594d581a1a40489d94c1393965a6f976'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.849081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594d581a1a40489d94c1393965a6f976'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.849737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594d581a1a40489d94c1393965a6f976'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.868628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594d581a1a40489d94c1393965a6f976'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.869563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a1d8fa8b-754b-4753-ab7b-192df6c353c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594d581a1a40489d94c1393965a6f976'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:02.883841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1046/2059] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:02 2022] DELETE /identity/v3/users/594d581a1a40489d94c1393965a6f976 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:03.319898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:03.335788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285e630-4185-42c1-98e5-ca1b9fa0e4df service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:03.421182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:03.422072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:03.422540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:03.423016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] PATH_INFO: `/v3/projects/effbfe24efda429d9a90343a2b71c48e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:03.425053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=effbfe24efda429d9a90343a2b71c48e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:03.427453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:03.512098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'effbfe24efda429d9a90343a2b71c48e', 'request_id': 'req-7da8220f-dbb9-478c-bf03-23e736f39916'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:03.519698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'effbfe24efda429d9a90343a2b71c48e', 'request_id': 'req-7da8220f-dbb9-478c-bf03-23e736f39916'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:03.532493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'effbfe24efda429d9a90343a2b71c48e', 'request_id': 'req-7da8220f-dbb9-478c-bf03-23e736f39916'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:03.548392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] The token cache is being invalidate because project effbfe24efda429d9a90343a2b71c48e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:03.548826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:03.549730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7da8220f-dbb9-478c-bf03-23e736f39916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:03.551888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1014/2060] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:03 2022] DELETE /identity/v3/projects/effbfe24efda429d9a90343a2b71c48e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:03.556980 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e2f72-59ef-4652-87b1-370fb47f4e54 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:03.557530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e2f72-59ef-4652-87b1-370fb47f4e54 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:03.557762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e2f72-59ef-4652-87b1-370fb47f4e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:03.604353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9a0e2f72-59ef-4652-87b1-370fb47f4e54 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:03.606287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9a0e2f72-59ef-4652-87b1-370fb47f4e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:03.688224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1047/2061] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:03.697454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0e66e-8070-450e-a5e8-d1efccb3ba15 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:03.697858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0e66e-8070-450e-a5e8-d1efccb3ba15 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:03.698236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0e66e-8070-450e-a5e8-d1efccb3ba15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:03.727167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-29b0e66e-8070-450e-a5e8-d1efccb3ba15 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:03.728444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-29b0e66e-8070-450e-a5e8-d1efccb3ba15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:03.737369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1015/2062] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:03.742342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e2f72-59ef-4652-87b1-370fb47f4e54 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:03.755207 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e2f72-59ef-4652-87b1-370fb47f4e54 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:03.760019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:03.760629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:03.760905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:03.761175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:03.761831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:03.762129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:03.763985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:03.778181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1048/2063] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:03.782605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0e66e-8070-450e-a5e8-d1efccb3ba15 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:03.793049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0e66e-8070-450e-a5e8-d1efccb3ba15 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:03.798262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1c90c-dea1-4b5a-9bcf-5351fa087f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:03.798917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1c90c-dea1-4b5a-9bcf-5351fa087f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:03.799207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1c90c-dea1-4b5a-9bcf-5351fa087f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:03.799475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1c90c-dea1-4b5a-9bcf-5351fa087f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:03.800128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1c90c-dea1-4b5a-9bcf-5351fa087f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:03.802064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1c90c-dea1-4b5a-9bcf-5351fa087f75 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:03.858509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1016/2064] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:03 2022] POST /identity/v3/projects => generated 363 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:03.862976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:03.895654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127280b-2f85-41f8-9845-36fc50790c3b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:03.955779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:04.012334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdeaf2-9000-479b-88ac-f52b5d93bc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.013150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdeaf2-9000-479b-88ac-f52b5d93bc88 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.013475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdeaf2-9000-479b-88ac-f52b5d93bc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.013821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdeaf2-9000-479b-88ac-f52b5d93bc88 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.014758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdeaf2-9000-479b-88ac-f52b5d93bc88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.016751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdeaf2-9000-479b-88ac-f52b5d93bc88 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.041052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1049/2065] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:03 2022] POST /identity/v3/users => generated 282 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:04.045486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1c90c-dea1-4b5a-9bcf-5351fa087f75 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.057068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1c90c-dea1-4b5a-9bcf-5351fa087f75 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.061906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65ee4b-2c22-45fb-beed-71dc10240bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.062568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ee4b-2c22-45fb-beed-71dc10240bab admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.062887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ee4b-2c22-45fb-beed-71dc10240bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.063170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ee4b-2c22-45fb-beed-71dc10240bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.063822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65ee4b-2c22-45fb-beed-71dc10240bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.065446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65ee4b-2c22-45fb-beed-71dc10240bab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.092506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1017/2066] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:04.093778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdeaf2-9000-479b-88ac-f52b5d93bc88 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.112721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdeaf2-9000-479b-88ac-f52b5d93bc88 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.119179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a3a60-1a55-4010-b14a-deb8cd9724ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.119899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a3a60-1a55-4010-b14a-deb8cd9724ee admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.120296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a3a60-1a55-4010-b14a-deb8cd9724ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.120622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a3a60-1a55-4010-b14a-deb8cd9724ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.121338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a3a60-1a55-4010-b14a-deb8cd9724ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.123329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a3a60-1a55-4010-b14a-deb8cd9724ee admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.136849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1050/2067] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:04.141858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65ee4b-2c22-45fb-beed-71dc10240bab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.153385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65ee4b-2c22-45fb-beed-71dc10240bab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.158431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.159131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.159406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.159672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] PATH_INFO: `/v3/projects/277093a2612942bca65f390f2b69c2e8/users/3deb079dd0774fa481d69864c3933337/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.176281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=277093a2612942bca65f390f2b69c2e8, user_id=3deb079dd0774fa481d69864c3933337, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.178984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.201818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1018/2068] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:04 2022] PUT /identity/v3/projects/277093a2612942bca65f390f2b69c2e8/users/3deb079dd0774fa481d69864c3933337/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:04.213431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac410e29-cdc8-4fcc-8f0f-7eeb1daa1bda None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.213720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac410e29-cdc8-4fcc-8f0f-7eeb1daa1bda None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.213995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac410e29-cdc8-4fcc-8f0f-7eeb1daa1bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.249882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ac410e29-cdc8-4fcc-8f0f-7eeb1daa1bda None None] MFA Rules not processed for user `3deb079dd0774fa481d69864c3933337`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:04.251244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ac410e29-cdc8-4fcc-8f0f-7eeb1daa1bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:04.267130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.269313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:04.277989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:04.340442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1051/2069] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:04.345122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac410e29-cdc8-4fcc-8f0f-7eeb1daa1bda None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.364233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e6b50-89d1-42bc-ac4a-ee79af7f56ee admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.379617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.380625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.381195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.381322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] PATH_INFO: `/v3/users/0f863757bba840ea85db119179b9fc7c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.382643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f863757bba840ea85db119179b9fc7c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.385044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.435288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac410e29-cdc8-4fcc-8f0f-7eeb1daa1bda None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.435288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bc1c2-5f9f-497f-92af-450e07be56f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.435288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc1c2-5f9f-497f-92af-450e07be56f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.435288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc1c2-5f9f-497f-92af-450e07be56f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.435728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc1c2-5f9f-497f-92af-450e07be56f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.435728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bc1c2-5f9f-497f-92af-450e07be56f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.435728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bc1c2-5f9f-497f-92af-450e07be56f4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.462100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f863757bba840ea85db119179b9fc7c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:04.468190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1052/2070] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:04 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:04.473026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bc1c2-5f9f-497f-92af-450e07be56f4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.475571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f863757bba840ea85db119179b9fc7c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:04.477796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f863757bba840ea85db119179b9fc7c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:04.493357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f863757bba840ea85db119179b9fc7c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:04.494269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f863757bba840ea85db119179b9fc7c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:04.501951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:04.511903 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1019/2071] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:04 2022] DELETE /identity/v3/users/0f863757bba840ea85db119179b9fc7c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:04.551758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bc1c2-5f9f-497f-92af-450e07be56f4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.624964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ef9d5-ffcc-44d5-936b-275cb6b840ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.625686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ef9d5-ffcc-44d5-936b-275cb6b840ec admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.626757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ef9d5-ffcc-44d5-936b-275cb6b840ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.626976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ef9d5-ffcc-44d5-936b-275cb6b840ec admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.627607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ef9d5-ffcc-44d5-936b-275cb6b840ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.629576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ef9d5-ffcc-44d5-936b-275cb6b840ec admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.661237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1053/2072] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:04 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:04.666461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.668340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:04.696031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ac5d1-ce35-45ca-97dd-566d77e9e7fe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.708172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8be170-2bee-4556-a41d-93eee8ac45e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.708816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be170-2bee-4556-a41d-93eee8ac45e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.709093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be170-2bee-4556-a41d-93eee8ac45e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.709361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8be170-2bee-4556-a41d-93eee8ac45e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.710104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8be170-2bee-4556-a41d-93eee8ac45e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.711650 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8be170-2bee-4556-a41d-93eee8ac45e8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.731231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1020/2073] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:04.735991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ef9d5-ffcc-44d5-936b-275cb6b840ec admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.747321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ef9d5-ffcc-44d5-936b-275cb6b840ec admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.752237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.753131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.753314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.753753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] PATH_INFO: `/v3/projects/91560859b3fc4c33a5bb63ae2b99b8eb/users/6f32fb616c1546349abde6da0b6d9007/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.775776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91560859b3fc4c33a5bb63ae2b99b8eb, user_id=6f32fb616c1546349abde6da0b6d9007, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.780848 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.798757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1054/2074] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:04 2022] PUT /identity/v3/projects/91560859b3fc4c33a5bb63ae2b99b8eb/users/6f32fb616c1546349abde6da0b6d9007/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:04.803287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8be170-2bee-4556-a41d-93eee8ac45e8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.813814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8be170-2bee-4556-a41d-93eee8ac45e8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:04.835186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:04.837098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:04.965402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cbe76-d617-4ab1-90f9-cb926b7c6f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:04.966286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cbe76-d617-4ab1-90f9-cb926b7c6f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:04.966746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cbe76-d617-4ab1-90f9-cb926b7c6f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:04.967163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cbe76-d617-4ab1-90f9-cb926b7c6f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:04.967960 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cbe76-d617-4ab1-90f9-cb926b7c6f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:04.969710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cbe76-d617-4ab1-90f9-cb926b7c6f2b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:04.988103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1021/2075] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:04.988932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cbe76-d617-4ab1-90f9-cb926b7c6f2b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.003294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b129e-3192-4411-b79e-deac3cd00465 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.006423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cbe76-d617-4ab1-90f9-cb926b7c6f2b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.008492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.009135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.009411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.009679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] PATH_INFO: `/v3/projects/2967ca7d73584b45baa158f5994f64df` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.011155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2967ca7d73584b45baa158f5994f64df)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.011699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664da08-36e7-4e20-b3c5-579ea5ed6366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.012445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664da08-36e7-4e20-b3c5-579ea5ed6366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.012788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664da08-36e7-4e20-b3c5-579ea5ed6366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.013291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.013624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664da08-36e7-4e20-b3c5-579ea5ed6366 admin admin] PATH_INFO: `/v3/projects/91560859b3fc4c33a5bb63ae2b99b8eb/users/6f32fb616c1546349abde6da0b6d9007/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.015698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664da08-36e7-4e20-b3c5-579ea5ed6366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91560859b3fc4c33a5bb63ae2b99b8eb, user_id=6f32fb616c1546349abde6da0b6d9007, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.018506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664da08-36e7-4e20-b3c5-579ea5ed6366 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.022085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:05.041451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1022/2076] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:04 2022] PUT /identity/v3/projects/91560859b3fc4c33a5bb63ae2b99b8eb/users/6f32fb616c1546349abde6da0b6d9007/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:05.047548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7691-05b7-49a0-9e67-0f27dbb88fef None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.047965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7691-05b7-49a0-9e67-0f27dbb88fef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.048380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7691-05b7-49a0-9e67-0f27dbb88fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.064001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:05.085841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-559b7691-05b7-49a0-9e67-0f27dbb88fef None None] MFA Rules not processed for user `6f32fb616c1546349abde6da0b6d9007`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:05.087080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-559b7691-05b7-49a0-9e67-0f27dbb88fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:05.088584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2967ca7d73584b45baa158f5994f64df', 'request_id': 'req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:05.099353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2967ca7d73584b45baa158f5994f64df', 'request_id': 'req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:05.106409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2967ca7d73584b45baa158f5994f64df', 'request_id': 'req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:05.116793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] The token cache is being invalidate because project 2967ca7d73584b45baa158f5994f64df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:05.117072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:05.117714 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d2d4cbd9-2ba0-467d-92fd-db4bcf72082e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:05.119257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1055/2077] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:04 2022] DELETE /identity/v3/projects/2967ca7d73584b45baa158f5994f64df => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:05.124415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730de8fe-6bb6-4f6e-9260-2d79bc192fc0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.124694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730de8fe-6bb6-4f6e-9260-2d79bc192fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.124966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730de8fe-6bb6-4f6e-9260-2d79bc192fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.143992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-730de8fe-6bb6-4f6e-9260-2d79bc192fc0 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:05.145113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-730de8fe-6bb6-4f6e-9260-2d79bc192fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:05.167043 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1023/2078] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:05.180225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7691-05b7-49a0-9e67-0f27dbb88fef None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.181610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-559b7691-05b7-49a0-9e67-0f27dbb88fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:05.209146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1056/2079] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:05.217551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c023-11d5-458d-93c4-9f94c2f33f2b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.218041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c023-11d5-458d-93c4-9f94c2f33f2b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.218315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c023-11d5-458d-93c4-9f94c2f33f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.239590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2c30c023-11d5-458d-93c4-9f94c2f33f2b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:05.240417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7691-05b7-49a0-9e67-0f27dbb88fef None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.240777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2c30c023-11d5-458d-93c4-9f94c2f33f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:05.244103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ed97e-2a34-4bfa-aaa3-ac64da464f4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.244704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ed97e-2a34-4bfa-aaa3-ac64da464f4d service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.244988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ed97e-2a34-4bfa-aaa3-ac64da464f4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.245254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ed97e-2a34-4bfa-aaa3-ac64da464f4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.248851 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1057/2080] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:05.253062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30c023-11d5-458d-93c4-9f94c2f33f2b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.254672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ed97e-2a34-4bfa-aaa3-ac64da464f4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.257049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ed97e-2a34-4bfa-aaa3-ac64da464f4d service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.262300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30c023-11d5-458d-93c4-9f94c2f33f2b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.265998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1024/2081] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:36:05.267339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.267959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.268202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.268466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.269125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:05.269417 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.271069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.283692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1058/2082] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:36:05.288169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ed97e-2a34-4bfa-aaa3-ac64da464f4d service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.301018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ed97e-2a34-4bfa-aaa3-ac64da464f4d service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.305678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bf2de-30c8-42c7-96be-1716e1d2cb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.306282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bf2de-30c8-42c7-96be-1716e1d2cb58 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.306567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bf2de-30c8-42c7-96be-1716e1d2cb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.306914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bf2de-30c8-42c7-96be-1716e1d2cb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.307559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bf2de-30c8-42c7-96be-1716e1d2cb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.309966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bf2de-30c8-42c7-96be-1716e1d2cb58 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.341587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1025/2083] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:05 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:05.346842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.362278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509c0d5-7e42-4950-9378-daa65a4e402e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.372442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bf2de-30c8-42c7-96be-1716e1d2cb58 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.426216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a0901-6a11-4521-bbc1-f2194a968a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.426885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0901-6a11-4521-bbc1-f2194a968a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.427167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0901-6a11-4521-bbc1-f2194a968a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.427443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0901-6a11-4521-bbc1-f2194a968a92 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.428112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a0901-6a11-4521-bbc1-f2194a968a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.430170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a0901-6a11-4521-bbc1-f2194a968a92 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.432519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bf2de-30c8-42c7-96be-1716e1d2cb58 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.438262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.438963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.439243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.439511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.441483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:05.452951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1059/2084] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:05 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:05.457126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:05.457437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a0901-6a11-4521-bbc1-f2194a968a92 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.468343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a0901-6a11-4521-bbc1-f2194a968a92 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.473425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff178e-bed8-4f78-b41b-123cda5ae02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.474037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff178e-bed8-4f78-b41b-123cda5ae02d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.474354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff178e-bed8-4f78-b41b-123cda5ae02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.474593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff178e-bed8-4f78-b41b-123cda5ae02d admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.475549 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff178e-bed8-4f78-b41b-123cda5ae02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.477170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff178e-bed8-4f78-b41b-123cda5ae02d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.488316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1060/2085] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:05.492305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff178e-bed8-4f78-b41b-123cda5ae02d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.501935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff178e-bed8-4f78-b41b-123cda5ae02d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.506320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.507383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676c417-da88-41d6-9a5f-28f669857d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.507827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.507957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676c417-da88-41d6-9a5f-28f669857d43 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.508260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676c417-da88-41d6-9a5f-28f669857d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.508653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676c417-da88-41d6-9a5f-28f669857d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.509297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676c417-da88-41d6-9a5f-28f669857d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.511056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676c417-da88-41d6-9a5f-28f669857d43 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.517099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1026/2086] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:05.521524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1061/2087] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:05.525701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.537273 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a961354-e63d-4784-905c-95f952b7ac29 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.542418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cbc35-7a15-4f3c-a74d-73629aaa9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.543132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cbc35-7a15-4f3c-a74d-73629aaa9608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.543434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cbc35-7a15-4f3c-a74d-73629aaa9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.543740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cbc35-7a15-4f3c-a74d-73629aaa9608 admin admin] PATH_INFO: `/v3/projects/f854aca016b8429ba1adf20569071c11/users/3c58d83217bc430b87c410aa3a778423/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.563784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cbc35-7a15-4f3c-a74d-73629aaa9608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f854aca016b8429ba1adf20569071c11, user_id=3c58d83217bc430b87c410aa3a778423, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.567078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cbc35-7a15-4f3c-a74d-73629aaa9608 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.581615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1027/2088] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:05 2022] PUT /identity/v3/projects/f854aca016b8429ba1adf20569071c11/users/3c58d83217bc430b87c410aa3a778423/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:05.593283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9243-d1d8-4122-b48e-33ef753693b4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.593674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9243-d1d8-4122-b48e-33ef753693b4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.594050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9243-d1d8-4122-b48e-33ef753693b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.609139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:05.631266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d79d9243-d1d8-4122-b48e-33ef753693b4 None None] MFA Rules not processed for user `3c58d83217bc430b87c410aa3a778423`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:05.632653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d79d9243-d1d8-4122-b48e-33ef753693b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:05.704491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1062/2089] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:05.709123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cbc35-7a15-4f3c-a74d-73629aaa9608 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.793734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cbc35-7a15-4f3c-a74d-73629aaa9608 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.798417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cc37c-17ed-4e90-8f12-60ce796bfb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.799056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cc37c-17ed-4e90-8f12-60ce796bfb2e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.799333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cc37c-17ed-4e90-8f12-60ce796bfb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.799597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cc37c-17ed-4e90-8f12-60ce796bfb2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.800230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cc37c-17ed-4e90-8f12-60ce796bfb2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.802106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cc37c-17ed-4e90-8f12-60ce796bfb2e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.830739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1028/2090] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:05 2022] POST /identity/v3/projects => generated 361 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:05.834972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9243-d1d8-4122-b48e-33ef753693b4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.845695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9243-d1d8-4122-b48e-33ef753693b4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.905749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfbe63-41fb-4e9d-b9f7-1afed3f120d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.906365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfbe63-41fb-4e9d-b9f7-1afed3f120d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.906640 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfbe63-41fb-4e9d-b9f7-1afed3f120d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.906940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfbe63-41fb-4e9d-b9f7-1afed3f120d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.907587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfbe63-41fb-4e9d-b9f7-1afed3f120d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.909664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfbe63-41fb-4e9d-b9f7-1afed3f120d0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.933662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1063/2091] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:05 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:05.938300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cc37c-17ed-4e90-8f12-60ce796bfb2e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.948684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cc37c-17ed-4e90-8f12-60ce796bfb2e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.954081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49096f4-a099-4f7a-b3fc-7d47c104c4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:05.954843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49096f4-a099-4f7a-b3fc-7d47c104c4aa admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:05.955051 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49096f4-a099-4f7a-b3fc-7d47c104c4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:05.955314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49096f4-a099-4f7a-b3fc-7d47c104c4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:05.955956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49096f4-a099-4f7a-b3fc-7d47c104c4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:05.958588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49096f4-a099-4f7a-b3fc-7d47c104c4aa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:05.972969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1029/2092] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:05.978530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfbe63-41fb-4e9d-b9f7-1afed3f120d0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:05.993058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfbe63-41fb-4e9d-b9f7-1afed3f120d0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:05.999118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4c9cd-4662-4a01-acfb-c37c3f59c9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:06.000046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4c9cd-4662-4a01-acfb-c37c3f59c9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:06.000322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4c9cd-4662-4a01-acfb-c37c3f59c9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:06.000625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4c9cd-4662-4a01-acfb-c37c3f59c9b8 admin admin] PATH_INFO: `/v3/projects/3427757f940a489fa106ba72ad87b420/users/606a838984354775adf0441d5f544c54/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:06.034917 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4c9cd-4662-4a01-acfb-c37c3f59c9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3427757f940a489fa106ba72ad87b420, user_id=606a838984354775adf0441d5f544c54, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:06.036033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4c9cd-4662-4a01-acfb-c37c3f59c9b8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:06.058290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1064/2093] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:05 2022] PUT /identity/v3/projects/3427757f940a489fa106ba72ad87b420/users/606a838984354775adf0441d5f544c54/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:06.061040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49096f4-a099-4f7a-b3fc-7d47c104c4aa admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:06.076622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49096f4-a099-4f7a-b3fc-7d47c104c4aa admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:06.154230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e81635-a550-4e7d-be24-2664a6be4b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:06.154889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e81635-a550-4e7d-be24-2664a6be4b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:06.155169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e81635-a550-4e7d-be24-2664a6be4b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:06.155437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e81635-a550-4e7d-be24-2664a6be4b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:06.156095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e81635-a550-4e7d-be24-2664a6be4b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:06.157730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e81635-a550-4e7d-be24-2664a6be4b95 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:06.168585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1030/2094] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:06.173152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4c9cd-4662-4a01-acfb-c37c3f59c9b8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:06.184053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4c9cd-4662-4a01-acfb-c37c3f59c9b8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:06.189685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83a64f-d889-45be-8284-6146d6a275e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:06.191498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83a64f-d889-45be-8284-6146d6a275e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:06.191790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83a64f-d889-45be-8284-6146d6a275e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:06.192068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83a64f-d889-45be-8284-6146d6a275e3 admin admin] PATH_INFO: `/v3/projects/3427757f940a489fa106ba72ad87b420/users/606a838984354775adf0441d5f544c54/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:06.194229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83a64f-d889-45be-8284-6146d6a275e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3427757f940a489fa106ba72ad87b420, user_id=606a838984354775adf0441d5f544c54, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:06.196930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83a64f-d889-45be-8284-6146d6a275e3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:06.208855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1065/2095] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:06 2022] PUT /identity/v3/projects/3427757f940a489fa106ba72ad87b420/users/606a838984354775adf0441d5f544c54/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:06.221483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d6f1d-82f3-4157-9b95-7acf136c5d32 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:06.221851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d6f1d-82f3-4157-9b95-7acf136c5d32 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:06.222142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d6f1d-82f3-4157-9b95-7acf136c5d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:06.230897 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:06.258879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bb0d6f1d-82f3-4157-9b95-7acf136c5d32 None None] MFA Rules not processed for user `606a838984354775adf0441d5f544c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:06.260129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bb0d6f1d-82f3-4157-9b95-7acf136c5d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:06.345343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1031/2096] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:06.357947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83a64f-d889-45be-8284-6146d6a275e3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:06.415029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:06.419894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83a64f-d889-45be-8284-6146d6a275e3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:06.425365 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a3e12-ec55-480b-8bb7-1db9de3494ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:06.425987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a3e12-ec55-480b-8bb7-1db9de3494ba service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:06.426373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a3e12-ec55-480b-8bb7-1db9de3494ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:06.426956 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a3e12-ec55-480b-8bb7-1db9de3494ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:06.438112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a3e12-ec55-480b-8bb7-1db9de3494ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:06.439592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a3e12-ec55-480b-8bb7-1db9de3494ba service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:06.449578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1066/2097] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:06 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:06.528168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d6f1d-82f3-4157-9b95-7acf136c5d32 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:06.539584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d6f1d-82f3-4157-9b95-7acf136c5d32 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:06.543643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1a612-104d-4571-bc2e-c61f24285fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:06.544358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1a612-104d-4571-bc2e-c61f24285fce service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:06.544639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1a612-104d-4571-bc2e-c61f24285fce service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:06.545056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1a612-104d-4571-bc2e-c61f24285fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:06.608396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1a612-104d-4571-bc2e-c61f24285fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:06.609920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1a612-104d-4571-bc2e-c61f24285fce service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:06.620205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1032/2098] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:09.417204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a3e12-ec55-480b-8bb7-1db9de3494ba service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:09.484251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a3e12-ec55-480b-8bb7-1db9de3494ba service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:09.490258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:09.490951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:09.491226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:09.491496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] PATH_INFO: `/v3/users/3c58d83217bc430b87c410aa3a778423` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:09.492791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c58d83217bc430b87c410aa3a778423)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:09.494740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:09.546643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c58d83217bc430b87c410aa3a778423'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:09.554327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c58d83217bc430b87c410aa3a778423'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:09.555079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c58d83217bc430b87c410aa3a778423'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:09.571239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c58d83217bc430b87c410aa3a778423'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:09.572062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c58d83217bc430b87c410aa3a778423'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:09.587248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1067/2099] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:09 2022] DELETE /identity/v3/users/3c58d83217bc430b87c410aa3a778423 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:09.593204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e264f-fd25-4687-b872-40213403c2cf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:09.593735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e264f-fd25-4687-b872-40213403c2cf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:09.593813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e264f-fd25-4687-b872-40213403c2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:09.614708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6d0e264f-fd25-4687-b872-40213403c2cf None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:09.616037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d0e264f-fd25-4687-b872-40213403c2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:09.680181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1033/2100] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:09.696890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:09.698389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:09.768337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98937258-96b5-4510-9401-7e9953358a9d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:09.772314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f8d0-b51e-438c-8582-00ec459539a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:09.772909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f8d0-b51e-438c-8582-00ec459539a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:09.773178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f8d0-b51e-438c-8582-00ec459539a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:09.773495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f8d0-b51e-438c-8582-00ec459539a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:09.783079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2f8d0-b51e-438c-8582-00ec459539a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:09.784537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2f8d0-b51e-438c-8582-00ec459539a4 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:09.794250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1068/2101] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:10.102983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e264f-fd25-4687-b872-40213403c2cf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:10.104445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d0e264f-fd25-4687-b872-40213403c2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:10.116822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e264f-fd25-4687-b872-40213403c2cf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:10.122251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:10.123667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:10.123952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:10.124224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] PATH_INFO: `/v3/projects/f854aca016b8429ba1adf20569071c11` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:10.126159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f854aca016b8429ba1adf20569071c11)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:10.128126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:10.154377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:10.182187 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f854aca016b8429ba1adf20569071c11', 'request_id': 'req-2b0b9d71-1196-4212-a792-4648345d345f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.189018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f854aca016b8429ba1adf20569071c11', 'request_id': 'req-2b0b9d71-1196-4212-a792-4648345d345f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.197093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f854aca016b8429ba1adf20569071c11', 'request_id': 'req-2b0b9d71-1196-4212-a792-4648345d345f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.207758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] The token cache is being invalidate because project f854aca016b8429ba1adf20569071c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:10.208040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.208680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.210208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1034/2102] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:10 2022] DELETE /identity/v3/projects/f854aca016b8429ba1adf20569071c11 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:10.214535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f8d0-b51e-438c-8582-00ec459539a4 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:10.225366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f8d0-b51e-438c-8582-00ec459539a4 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:10.235752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:10.295930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:10.296726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:10.297010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:10.297311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] PATH_INFO: `/v3/users/606a838984354775adf0441d5f544c54` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:10.298493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=606a838984354775adf0441d5f544c54)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:10.300510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:10.349954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '606a838984354775adf0441d5f544c54'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.358652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '606a838984354775adf0441d5f544c54'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.359322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '606a838984354775adf0441d5f544c54'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.372042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '606a838984354775adf0441d5f544c54'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.372911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a25ad707-d4db-45b7-ae41-ee6efcfd0f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '606a838984354775adf0441d5f544c54'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.388104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1069/2103] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:10 2022] DELETE /identity/v3/users/606a838984354775adf0441d5f544c54 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:10.689633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:10.691223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:10.706977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b9d71-1196-4212-a792-4648345d345f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:10.773446 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:10.774088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:10.774364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:10.774628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] PATH_INFO: `/v3/projects/3427757f940a489fa106ba72ad87b420` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:10.775813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3427757f940a489fa106ba72ad87b420)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:10.777808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:10.844770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3427757f940a489fa106ba72ad87b420', 'request_id': 'req-5dd458b0-4597-445d-8615-22f93efd5976'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.853908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3427757f940a489fa106ba72ad87b420', 'request_id': 'req-5dd458b0-4597-445d-8615-22f93efd5976'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.865129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3427757f940a489fa106ba72ad87b420', 'request_id': 'req-5dd458b0-4597-445d-8615-22f93efd5976'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.877264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] The token cache is being invalidate because project 3427757f940a489fa106ba72ad87b420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:10.877579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.882700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5dd458b0-4597-445d-8615-22f93efd5976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:10.884471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1035/2104] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:10 2022] DELETE /identity/v3/projects/3427757f940a489fa106ba72ad87b420 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:10.889887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05725b36-5aad-447c-9a72-72003ac33b88 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:10.890363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05725b36-5aad-447c-9a72-72003ac33b88 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:10.890439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05725b36-5aad-447c-9a72-72003ac33b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:10.909784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-05725b36-5aad-447c-9a72-72003ac33b88 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:10.910987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-05725b36-5aad-447c-9a72-72003ac33b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:10.976448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1070/2105] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:10.985057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826d3e8-c187-46fd-910e-aeb61e1f3bd9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:10.985338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826d3e8-c187-46fd-910e-aeb61e1f3bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:10.985680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826d3e8-c187-46fd-910e-aeb61e1f3bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.009212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c826d3e8-c187-46fd-910e-aeb61e1f3bd9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:11.010641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c826d3e8-c187-46fd-910e-aeb61e1f3bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:11.023964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1036/2106] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:11.028214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05725b36-5aad-447c-9a72-72003ac33b88 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.038983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05725b36-5aad-447c-9a72-72003ac33b88 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.044856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.045479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.045853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.046014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.046674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:11.047023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.059238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.064533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1071/2107] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:11.069143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826d3e8-c187-46fd-910e-aeb61e1f3bd9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.079278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826d3e8-c187-46fd-910e-aeb61e1f3bd9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.084882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbd156-3ce7-4a6a-b4aa-befdbaf522e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.085519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbd156-3ce7-4a6a-b4aa-befdbaf522e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.085793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbd156-3ce7-4a6a-b4aa-befdbaf522e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.086057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbd156-3ce7-4a6a-b4aa-befdbaf522e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.086687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbd156-3ce7-4a6a-b4aa-befdbaf522e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.088667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbd156-3ce7-4a6a-b4aa-befdbaf522e4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.117519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1037/2108] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:11 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:11.122023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.133609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0a217-54ff-4617-8656-a42c08dde212 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.152856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:11.193449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e3a8c-5304-4662-8f55-fa9cffd6ee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.194067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e3a8c-5304-4662-8f55-fa9cffd6ee78 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.194345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e3a8c-5304-4662-8f55-fa9cffd6ee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.194611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e3a8c-5304-4662-8f55-fa9cffd6ee78 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.195271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e3a8c-5304-4662-8f55-fa9cffd6ee78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.197208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e3a8c-5304-4662-8f55-fa9cffd6ee78 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.220195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1072/2109] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:11 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:11.224769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbd156-3ce7-4a6a-b4aa-befdbaf522e4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.236636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbd156-3ce7-4a6a-b4aa-befdbaf522e4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.241773 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617aafc-d9e4-4fa7-82fc-0de86ae4014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.242417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617aafc-d9e4-4fa7-82fc-0de86ae4014b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.242673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617aafc-d9e4-4fa7-82fc-0de86ae4014b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.242980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617aafc-d9e4-4fa7-82fc-0de86ae4014b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.243628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617aafc-d9e4-4fa7-82fc-0de86ae4014b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.245267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617aafc-d9e4-4fa7-82fc-0de86ae4014b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.252155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:11.260350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1038/2110] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:11.261564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e3a8c-5304-4662-8f55-fa9cffd6ee78 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.272223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e3a8c-5304-4662-8f55-fa9cffd6ee78 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.277615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f80965-cb36-41c2-af41-1da0161f1354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.278337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f80965-cb36-41c2-af41-1da0161f1354 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.278640 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f80965-cb36-41c2-af41-1da0161f1354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.279043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f80965-cb36-41c2-af41-1da0161f1354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.279768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f80965-cb36-41c2-af41-1da0161f1354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.281445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f80965-cb36-41c2-af41-1da0161f1354 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.292953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1073/2111] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:11.297338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617aafc-d9e4-4fa7-82fc-0de86ae4014b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.307801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617aafc-d9e4-4fa7-82fc-0de86ae4014b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.316555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86403b0e-a16f-4d37-924d-ee3a18380284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.317244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86403b0e-a16f-4d37-924d-ee3a18380284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.317580 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86403b0e-a16f-4d37-924d-ee3a18380284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.317875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86403b0e-a16f-4d37-924d-ee3a18380284 admin admin] PATH_INFO: `/v3/projects/d8b81959343f415585fe1f45d4674af1/users/f59bb2c99cf14652a852d25827286157/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.342146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86403b0e-a16f-4d37-924d-ee3a18380284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b81959343f415585fe1f45d4674af1, user_id=f59bb2c99cf14652a852d25827286157, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.344879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86403b0e-a16f-4d37-924d-ee3a18380284 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.363811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1039/2112] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:11 2022] PUT /identity/v3/projects/d8b81959343f415585fe1f45d4674af1/users/f59bb2c99cf14652a852d25827286157/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:11.374904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351e9fe-ffa9-4914-8743-06bf58f7c84a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.375112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351e9fe-ffa9-4914-8743-06bf58f7c84a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.375381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351e9fe-ffa9-4914-8743-06bf58f7c84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.402777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f351e9fe-ffa9-4914-8743-06bf58f7c84a None None] MFA Rules not processed for user `f59bb2c99cf14652a852d25827286157`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:11.404063 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f351e9fe-ffa9-4914-8743-06bf58f7c84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:11.494072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1074/2113] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:11.499309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86403b0e-a16f-4d37-924d-ee3a18380284 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.565111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86403b0e-a16f-4d37-924d-ee3a18380284 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.572466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab1613-3684-4bd3-80a9-568e5892eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.573365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab1613-3684-4bd3-80a9-568e5892eff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.573813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab1613-3684-4bd3-80a9-568e5892eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.574262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab1613-3684-4bd3-80a9-568e5892eff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.574985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab1613-3684-4bd3-80a9-568e5892eff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.577330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab1613-3684-4bd3-80a9-568e5892eff8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.610329 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1040/2114] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:11 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:11.614893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351e9fe-ffa9-4914-8743-06bf58f7c84a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.629017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351e9fe-ffa9-4914-8743-06bf58f7c84a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.701598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c321ac-0a89-4325-9d8e-e3635e26a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.702192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c321ac-0a89-4325-9d8e-e3635e26a0fb admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.702465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c321ac-0a89-4325-9d8e-e3635e26a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.702753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c321ac-0a89-4325-9d8e-e3635e26a0fb admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.703395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c321ac-0a89-4325-9d8e-e3635e26a0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.705322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c321ac-0a89-4325-9d8e-e3635e26a0fb admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.729141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1075/2115] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:11 2022] POST /identity/v3/users => generated 301 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:11.733677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab1613-3684-4bd3-80a9-568e5892eff8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.744151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab1613-3684-4bd3-80a9-568e5892eff8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.749333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e40902-9b9b-46f4-9fff-92320dfc4b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.749996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e40902-9b9b-46f4-9fff-92320dfc4b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.750248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e40902-9b9b-46f4-9fff-92320dfc4b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.750517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e40902-9b9b-46f4-9fff-92320dfc4b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.751215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e40902-9b9b-46f4-9fff-92320dfc4b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.752782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e40902-9b9b-46f4-9fff-92320dfc4b1f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.764767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1041/2116] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:11.769321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c321ac-0a89-4325-9d8e-e3635e26a0fb admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.781793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c321ac-0a89-4325-9d8e-e3635e26a0fb admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.787686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2659a8-6cd8-4305-8331-e3bba4dd1c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.788400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2659a8-6cd8-4305-8331-e3bba4dd1c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.788704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2659a8-6cd8-4305-8331-e3bba4dd1c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.788978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2659a8-6cd8-4305-8331-e3bba4dd1c9d admin admin] PATH_INFO: `/v3/projects/a1d02cd36d9644e9932b95e658455c18/users/2dbd1ca737f4450ca8321df62cde35f2/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.804598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2659a8-6cd8-4305-8331-e3bba4dd1c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d02cd36d9644e9932b95e658455c18, user_id=2dbd1ca737f4450ca8321df62cde35f2, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.807555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2659a8-6cd8-4305-8331-e3bba4dd1c9d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.809719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:11.821346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1076/2117] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:11 2022] PUT /identity/v3/projects/a1d02cd36d9644e9932b95e658455c18/users/2dbd1ca737f4450ca8321df62cde35f2/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:11.825418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e40902-9b9b-46f4-9fff-92320dfc4b1f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.835596 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e40902-9b9b-46f4-9fff-92320dfc4b1f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.900085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea8739-add4-4a89-a400-8b8aef08b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.900700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea8739-add4-4a89-a400-8b8aef08b0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.900968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea8739-add4-4a89-a400-8b8aef08b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.901243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea8739-add4-4a89-a400-8b8aef08b0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.901879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea8739-add4-4a89-a400-8b8aef08b0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.903510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea8739-add4-4a89-a400-8b8aef08b0d3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.914251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1042/2118] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:11.918636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2659a8-6cd8-4305-8331-e3bba4dd1c9d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:11.929683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2659a8-6cd8-4305-8331-e3bba4dd1c9d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:11.934874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:11.935689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.935964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.936236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] PATH_INFO: `/v3/projects/a1d02cd36d9644e9932b95e658455c18/users/2dbd1ca737f4450ca8321df62cde35f2/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.938147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d02cd36d9644e9932b95e658455c18, user_id=2dbd1ca737f4450ca8321df62cde35f2, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:11.940911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:11.953222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1077/2119] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:11 2022] PUT /identity/v3/projects/a1d02cd36d9644e9932b95e658455c18/users/2dbd1ca737f4450ca8321df62cde35f2/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:11.965739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9605b3-a549-4a98-b88c-57da04fa12f1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:11.966024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9605b3-a549-4a98-b88c-57da04fa12f1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:11.966383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9605b3-a549-4a98-b88c-57da04fa12f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:11.997132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ad9605b3-a549-4a98-b88c-57da04fa12f1 None None] MFA Rules not processed for user `2dbd1ca737f4450ca8321df62cde35f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:11.998653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad9605b3-a549-4a98-b88c-57da04fa12f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:12.035100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:12.077581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1043/2120] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:12.088900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:12.090759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:12.153457 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732638e-21dd-4274-8222-b817229a31ef admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:12.157297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef8f50-f707-42db-b493-92f89c17925a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:12.157918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef8f50-f707-42db-b493-92f89c17925a service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:12.158197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef8f50-f707-42db-b493-92f89c17925a service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:12.158474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef8f50-f707-42db-b493-92f89c17925a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:12.170066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef8f50-f707-42db-b493-92f89c17925a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:12.171549 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef8f50-f707-42db-b493-92f89c17925a service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:12.181473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1078/2121] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:12 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:12.324333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9605b3-a549-4a98-b88c-57da04fa12f1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:12.392412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9605b3-a549-4a98-b88c-57da04fa12f1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:12.397519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:12.398091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:12.398370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:12.398708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] PATH_INFO: `/v3/users/f59bb2c99cf14652a852d25827286157` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:12.399816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59bb2c99cf14652a852d25827286157)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:12.401767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:12.456492 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59bb2c99cf14652a852d25827286157'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:12.464290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59bb2c99cf14652a852d25827286157'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:12.464878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59bb2c99cf14652a852d25827286157'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:12.477315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59bb2c99cf14652a852d25827286157'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:12.477971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59bb2c99cf14652a852d25827286157'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:12.490504 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1044/2122] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:12 2022] DELETE /identity/v3/users/f59bb2c99cf14652a852d25827286157 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:12.495307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c025f-9010-483a-b735-fe511828a988 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:12.495591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c025f-9010-483a-b735-fe511828a988 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:12.495862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c025f-9010-483a-b735-fe511828a988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:12.516878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-152c025f-9010-483a-b735-fe511828a988 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:12.518083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-152c025f-9010-483a-b735-fe511828a988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:12.580263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1079/2123] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:12.591429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:12.593116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:12.672710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b6340-070f-4330-bfe4-0266a7ba2a09 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:12.679120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a585b8-6cf8-4724-935c-2301291022f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:12.680157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a585b8-6cf8-4724-935c-2301291022f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:12.680645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a585b8-6cf8-4724-935c-2301291022f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:12.681176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a585b8-6cf8-4724-935c-2301291022f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:12.691961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:12.696811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a585b8-6cf8-4724-935c-2301291022f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:12.698514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a585b8-6cf8-4724-935c-2301291022f7 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:12.710054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1045/2124] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:13.004373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c025f-9010-483a-b735-fe511828a988 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:13.005866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-152c025f-9010-483a-b735-fe511828a988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:13.019699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c025f-9010-483a-b735-fe511828a988 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:13.025148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:13.025847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:13.026126 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:13.026393 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] PATH_INFO: `/v3/projects/d8b81959343f415585fe1f45d4674af1` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:13.027583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b81959343f415585fe1f45d4674af1)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:13.029555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:13.080375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b81959343f415585fe1f45d4674af1', 'request_id': 'req-fa4874b0-cf19-48e8-b643-ad7e6438b996'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.091730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b81959343f415585fe1f45d4674af1', 'request_id': 'req-fa4874b0-cf19-48e8-b643-ad7e6438b996'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.099638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b81959343f415585fe1f45d4674af1', 'request_id': 'req-fa4874b0-cf19-48e8-b643-ad7e6438b996'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.111129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] The token cache is being invalidate because project d8b81959343f415585fe1f45d4674af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:13.111410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.112092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.113626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1080/2125] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:13 2022] DELETE /identity/v3/projects/d8b81959343f415585fe1f45d4674af1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:13.117870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a585b8-6cf8-4724-935c-2301291022f7 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:13.129065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a585b8-6cf8-4724-935c-2301291022f7 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:13.221102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:13.222645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:13.223457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:13.223871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] PATH_INFO: `/v3/users/2dbd1ca737f4450ca8321df62cde35f2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:13.227842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dbd1ca737f4450ca8321df62cde35f2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:13.230101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:13.289204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dbd1ca737f4450ca8321df62cde35f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.301086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dbd1ca737f4450ca8321df62cde35f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.301805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dbd1ca737f4450ca8321df62cde35f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.320512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dbd1ca737f4450ca8321df62cde35f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.321262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04e8a943-30c8-48c6-953d-b9638f2d3542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dbd1ca737f4450ca8321df62cde35f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.337138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1046/2126] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:13 2022] DELETE /identity/v3/users/2dbd1ca737f4450ca8321df62cde35f2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:13.647211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:13.648702 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:13.664518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4874b0-cf19-48e8-b643-ad7e6438b996 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:13.672548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:13.732777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:13.733419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:13.733693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:13.733962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] PATH_INFO: `/v3/projects/a1d02cd36d9644e9932b95e658455c18` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:13.735128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d02cd36d9644e9932b95e658455c18)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:13.737070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:13.785476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d02cd36d9644e9932b95e658455c18', 'request_id': 'req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.794309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d02cd36d9644e9932b95e658455c18', 'request_id': 'req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.801183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d02cd36d9644e9932b95e658455c18', 'request_id': 'req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.812512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] The token cache is being invalidate because project a1d02cd36d9644e9932b95e658455c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:13.812799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.813561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-fe7d1766-1373-4eda-9f9c-7b3e8aed25f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:13.815222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1081/2127] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:13 2022] DELETE /identity/v3/projects/a1d02cd36d9644e9932b95e658455c18 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:13.823137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be9999-05b3-4071-82e5-b4d885d86b1b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:13.823687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be9999-05b3-4071-82e5-b4d885d86b1b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:13.824102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be9999-05b3-4071-82e5-b4d885d86b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:13.845655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-40be9999-05b3-4071-82e5-b4d885d86b1b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:13.847234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-40be9999-05b3-4071-82e5-b4d885d86b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:13.912882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1047/2128] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:13.921238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7b118-cde6-49c0-89bc-217a1d2f2bae None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:13.921497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7b118-cde6-49c0-89bc-217a1d2f2bae None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:13.921781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7b118-cde6-49c0-89bc-217a1d2f2bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:13.942396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-27d7b118-cde6-49c0-89bc-217a1d2f2bae None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:13.943647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-27d7b118-cde6-49c0-89bc-217a1d2f2bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:13.951960 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1082/2129] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:13.956365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be9999-05b3-4071-82e5-b4d885d86b1b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:13.966785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be9999-05b3-4071-82e5-b4d885d86b1b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:13.972197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:13.972796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:13.973068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:13.973337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:13.973986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:13.974285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:13.975911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:13.985270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:13.990397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1048/2130] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:13.994027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7b118-cde6-49c0-89bc-217a1d2f2bae None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.005303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7b118-cde6-49c0-89bc-217a1d2f2bae None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.010803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367cd05-5b7d-498b-b891-27ab5c913e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.011703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367cd05-5b7d-498b-b891-27ab5c913e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.011958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367cd05-5b7d-498b-b891-27ab5c913e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.012238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367cd05-5b7d-498b-b891-27ab5c913e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.012886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367cd05-5b7d-498b-b891-27ab5c913e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.014876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367cd05-5b7d-498b-b891-27ab5c913e07 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.044482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1083/2131] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:13 2022] POST /identity/v3/projects => generated 407 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:14.050470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.061211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef1868-1a7f-4c15-8887-b8a44e656e44 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.124705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdcd1e-136f-49c1-9c67-776e8f474d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.125326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdcd1e-136f-49c1-9c67-776e8f474d3d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.125598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdcd1e-136f-49c1-9c67-776e8f474d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.125865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdcd1e-136f-49c1-9c67-776e8f474d3d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.126524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdcd1e-136f-49c1-9c67-776e8f474d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.128448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdcd1e-136f-49c1-9c67-776e8f474d3d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.151452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1049/2132] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:14 2022] POST /identity/v3/users => generated 304 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:14.156246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367cd05-5b7d-498b-b891-27ab5c913e07 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.167381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367cd05-5b7d-498b-b891-27ab5c913e07 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.172763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f367d-dae3-4a65-948d-2d84f22a946c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.173469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f367d-dae3-4a65-948d-2d84f22a946c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.173748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f367d-dae3-4a65-948d-2d84f22a946c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.174142 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f367d-dae3-4a65-948d-2d84f22a946c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.174990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f367d-dae3-4a65-948d-2d84f22a946c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.176608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f367d-dae3-4a65-948d-2d84f22a946c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.187881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1084/2133] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:14.192191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdcd1e-136f-49c1-9c67-776e8f474d3d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.204999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdcd1e-136f-49c1-9c67-776e8f474d3d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.211257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5ed35-5686-4514-8b6a-f6414eaa714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.211920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ed35-5686-4514-8b6a-f6414eaa714f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.212258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ed35-5686-4514-8b6a-f6414eaa714f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.212470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5ed35-5686-4514-8b6a-f6414eaa714f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.213212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5ed35-5686-4514-8b6a-f6414eaa714f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.214866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5ed35-5686-4514-8b6a-f6414eaa714f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.226970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1050/2134] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:14.231788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f367d-dae3-4a65-948d-2d84f22a946c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.248291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f367d-dae3-4a65-948d-2d84f22a946c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.253663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e46b1c-d4c3-4f64-a618-02be24d8e15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.254470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e46b1c-d4c3-4f64-a618-02be24d8e15a admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.255039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e46b1c-d4c3-4f64-a618-02be24d8e15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.260584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e46b1c-d4c3-4f64-a618-02be24d8e15a admin admin] PATH_INFO: `/v3/projects/164d7a0a14fd483791eeb4acf65f52e5/users/7be7950a96534bdfa9a46cf4938a1c27/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.280934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e46b1c-d4c3-4f64-a618-02be24d8e15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=164d7a0a14fd483791eeb4acf65f52e5, user_id=7be7950a96534bdfa9a46cf4938a1c27, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.284444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e46b1c-d4c3-4f64-a618-02be24d8e15a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.299878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1085/2135] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:14 2022] PUT /identity/v3/projects/164d7a0a14fd483791eeb4acf65f52e5/users/7be7950a96534bdfa9a46cf4938a1c27/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:14.315026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96f644-c46d-4966-88e0-e17b097dfe1e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.315026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96f644-c46d-4966-88e0-e17b097dfe1e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.315026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96f644-c46d-4966-88e0-e17b097dfe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.375599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fb96f644-c46d-4966-88e0-e17b097dfe1e None None] MFA Rules not processed for user `7be7950a96534bdfa9a46cf4938a1c27`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:14.376957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb96f644-c46d-4966-88e0-e17b097dfe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:14.509379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1051/2136] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:14 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:14.514521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e46b1c-d4c3-4f64-a618-02be24d8e15a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.558424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:14.611114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e46b1c-d4c3-4f64-a618-02be24d8e15a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.616184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421181f-5394-4e40-8cef-30337d86cdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.616808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421181f-5394-4e40-8cef-30337d86cdfc admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.617086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421181f-5394-4e40-8cef-30337d86cdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.617378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421181f-5394-4e40-8cef-30337d86cdfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.618003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421181f-5394-4e40-8cef-30337d86cdfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.620002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421181f-5394-4e40-8cef-30337d86cdfc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.652152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1086/2137] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:14 2022] POST /identity/v3/projects => generated 409 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:14.656806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96f644-c46d-4966-88e0-e17b097dfe1e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.668233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96f644-c46d-4966-88e0-e17b097dfe1e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.736313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a089db7-5b83-4763-b777-9f3fae5ded2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.736942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089db7-5b83-4763-b777-9f3fae5ded2a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.737216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089db7-5b83-4763-b777-9f3fae5ded2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.737483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089db7-5b83-4763-b777-9f3fae5ded2a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.738116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a089db7-5b83-4763-b777-9f3fae5ded2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.740060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a089db7-5b83-4763-b777-9f3fae5ded2a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.757382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:14.767257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1052/2138] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:14 2022] POST /identity/v3/users => generated 311 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:14.770382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421181f-5394-4e40-8cef-30337d86cdfc admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.784401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421181f-5394-4e40-8cef-30337d86cdfc admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.790623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f7dad-e429-4de5-b1e7-8fb6e99e8ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.791584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f7dad-e429-4de5-b1e7-8fb6e99e8ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.791927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f7dad-e429-4de5-b1e7-8fb6e99e8ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.792289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f7dad-e429-4de5-b1e7-8fb6e99e8ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.793060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f7dad-e429-4de5-b1e7-8fb6e99e8ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.795357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f7dad-e429-4de5-b1e7-8fb6e99e8ce8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.806824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1087/2139] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:14.811349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a089db7-5b83-4763-b777-9f3fae5ded2a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.821512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a089db7-5b83-4763-b777-9f3fae5ded2a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.826657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8aa5-44e5-421d-a0c0-d8ae0746e8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.827334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8aa5-44e5-421d-a0c0-d8ae0746e8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.827606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8aa5-44e5-421d-a0c0-d8ae0746e8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.827874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8aa5-44e5-421d-a0c0-d8ae0746e8bf admin admin] PATH_INFO: `/v3/projects/79bb0cc9213247e6a470a86676aee4c1/users/50e295110ad643c8a3dbfb4ec1e21548/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.843484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa8aa5-44e5-421d-a0c0-d8ae0746e8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bb0cc9213247e6a470a86676aee4c1, user_id=50e295110ad643c8a3dbfb4ec1e21548, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.846335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa8aa5-44e5-421d-a0c0-d8ae0746e8bf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.860502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1053/2140] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:14 2022] PUT /identity/v3/projects/79bb0cc9213247e6a470a86676aee4c1/users/50e295110ad643c8a3dbfb4ec1e21548/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:14.866029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f7dad-e429-4de5-b1e7-8fb6e99e8ce8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.877760 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f7dad-e429-4de5-b1e7-8fb6e99e8ce8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.945552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9eeb8-2540-44db-bbdb-5010df52310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.946164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9eeb8-2540-44db-bbdb-5010df52310c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.946443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9eeb8-2540-44db-bbdb-5010df52310c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.946726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9eeb8-2540-44db-bbdb-5010df52310c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.947389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9eeb8-2540-44db-bbdb-5010df52310c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.949048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9eeb8-2540-44db-bbdb-5010df52310c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:14.959808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1088/2141] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:14.964472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8aa5-44e5-421d-a0c0-d8ae0746e8bf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:14.976419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8aa5-44e5-421d-a0c0-d8ae0746e8bf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:14.982079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:14.982821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:14.982970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:14.983209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] PATH_INFO: `/v3/projects/79bb0cc9213247e6a470a86676aee4c1/users/50e295110ad643c8a3dbfb4ec1e21548/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:14.985444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bb0cc9213247e6a470a86676aee4c1, user_id=50e295110ad643c8a3dbfb4ec1e21548, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:14.988257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:15.000565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1054/2142] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:14 2022] PUT /identity/v3/projects/79bb0cc9213247e6a470a86676aee4c1/users/50e295110ad643c8a3dbfb4ec1e21548/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:15.011437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6cde-23df-4b2f-ae07-8421736c3a0e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:15.011932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6cde-23df-4b2f-ae07-8421736c3a0e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:15.012022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6cde-23df-4b2f-ae07-8421736c3a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:15.041443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6eec6cde-23df-4b2f-ae07-8421736c3a0e None None] MFA Rules not processed for user `50e295110ad643c8a3dbfb4ec1e21548`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:15.042658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6eec6cde-23df-4b2f-ae07-8421736c3a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:15.120122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1089/2143] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:15 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:15.131557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:15.133473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:15.198378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067f142-745f-404d-ba37-7c1a6de32071 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:15.202795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:15.203431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:15.203734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:15.204028 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:15.214697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:15.216184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:15.229094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1055/2144] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:15 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:15.390249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6cde-23df-4b2f-ae07-8421736c3a0e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:15.401232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6cde-23df-4b2f-ae07-8421736c3a0e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:15.405270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badf9a1-2333-40f7-a0ae-62dc1dbce249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:15.405862 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badf9a1-2333-40f7-a0ae-62dc1dbce249 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:15.406137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badf9a1-2333-40f7-a0ae-62dc1dbce249 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:15.406399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badf9a1-2333-40f7-a0ae-62dc1dbce249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:15.433696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:15.474861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badf9a1-2333-40f7-a0ae-62dc1dbce249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:15.476315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badf9a1-2333-40f7-a0ae-62dc1dbce249 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:15.486212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1090/2145] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:15 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:23.799618 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:23.801499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:23.868246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f4dd8-8665-4bbf-9052-4d77ac3612d9 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:23.873083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:23.873715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:23.873988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:23.874396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] PATH_INFO: `/v3/users/3deb079dd0774fa481d69864c3933337` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:23.875865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3deb079dd0774fa481d69864c3933337)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:23.877779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:23.954532 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3deb079dd0774fa481d69864c3933337'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:23.967690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3deb079dd0774fa481d69864c3933337'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:23.968634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3deb079dd0774fa481d69864c3933337'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:23.978591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:23.987906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3deb079dd0774fa481d69864c3933337'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:23.989768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3deb079dd0774fa481d69864c3933337'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.012474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1056/2146] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:23 2022] DELETE /identity/v3/users/3deb079dd0774fa481d69864c3933337 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:24.017186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b19ea-03b0-42ee-9e6c-0f04daf96682 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:24.018101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b19ea-03b0-42ee-9e6c-0f04daf96682 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:24.018101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b19ea-03b0-42ee-9e6c-0f04daf96682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:24.043469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f18b19ea-03b0-42ee-9e6c-0f04daf96682 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:24.044792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f18b19ea-03b0-42ee-9e6c-0f04daf96682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:24.142696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1091/2147] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:24.154138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:24.155843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:24.220637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8e1b-09da-4909-8bc7-9b07477b3fc2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:24.224557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb67e9e-a07c-4742-ab2f-bad8fbf43285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:24.225185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb67e9e-a07c-4742-ab2f-bad8fbf43285 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:24.225444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb67e9e-a07c-4742-ab2f-bad8fbf43285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:24.225827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb67e9e-a07c-4742-ab2f-bad8fbf43285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:24.235179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb67e9e-a07c-4742-ab2f-bad8fbf43285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:24.236641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb67e9e-a07c-4742-ab2f-bad8fbf43285 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:24.246525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1057/2148] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:24.548390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b19ea-03b0-42ee-9e6c-0f04daf96682 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:24.550453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f18b19ea-03b0-42ee-9e6c-0f04daf96682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:24.565440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b19ea-03b0-42ee-9e6c-0f04daf96682 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:24.571734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:24.572429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:24.572647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:24.573029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] PATH_INFO: `/v3/projects/277093a2612942bca65f390f2b69c2e8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:24.574129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=277093a2612942bca65f390f2b69c2e8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:24.576168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:24.625627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277093a2612942bca65f390f2b69c2e8', 'request_id': 'req-6d519fc9-45f2-4fff-9768-1405b77b50ee'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.633961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277093a2612942bca65f390f2b69c2e8', 'request_id': 'req-6d519fc9-45f2-4fff-9768-1405b77b50ee'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.640612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277093a2612942bca65f390f2b69c2e8', 'request_id': 'req-6d519fc9-45f2-4fff-9768-1405b77b50ee'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.650821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] The token cache is being invalidate because project 277093a2612942bca65f390f2b69c2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:24.651161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.651729 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.653486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1092/2149] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:24 2022] DELETE /identity/v3/projects/277093a2612942bca65f390f2b69c2e8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:24.657486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb67e9e-a07c-4742-ab2f-bad8fbf43285 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:24.668208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb67e9e-a07c-4742-ab2f-bad8fbf43285 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:24.742365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:24.743119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:24.743395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:24.743660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] PATH_INFO: `/v3/users/6f32fb616c1546349abde6da0b6d9007` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:24.745407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f32fb616c1546349abde6da0b6d9007)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:24.747493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:24.811652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f32fb616c1546349abde6da0b6d9007'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.820351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f32fb616c1546349abde6da0b6d9007'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.821036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f32fb616c1546349abde6da0b6d9007'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.835789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f32fb616c1546349abde6da0b6d9007'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.836528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2c4eca8-77a9-4c01-907c-c4c158cb8d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f32fb616c1546349abde6da0b6d9007'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:24.849255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1058/2150] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:24 2022] DELETE /identity/v3/users/6f32fb616c1546349abde6da0b6d9007 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:25.144901 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:25.146612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:25.166647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d519fc9-45f2-4fff-9768-1405b77b50ee admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:25.244683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:25.267602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:25.268779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.269291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.269782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] PATH_INFO: `/v3/projects/91560859b3fc4c33a5bb63ae2b99b8eb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.272113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91560859b3fc4c33a5bb63ae2b99b8eb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:25.274391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:25.334763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91560859b3fc4c33a5bb63ae2b99b8eb', 'request_id': 'req-92790ff2-aa0c-4c18-b17f-108e0135ad23'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:25.344244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91560859b3fc4c33a5bb63ae2b99b8eb', 'request_id': 'req-92790ff2-aa0c-4c18-b17f-108e0135ad23'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:25.356132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91560859b3fc4c33a5bb63ae2b99b8eb', 'request_id': 'req-92790ff2-aa0c-4c18-b17f-108e0135ad23'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:25.368194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] The token cache is being invalidate because project 91560859b3fc4c33a5bb63ae2b99b8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:25.368607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:25.369470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92790ff2-aa0c-4c18-b17f-108e0135ad23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:25.371258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1093/2151] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:25 2022] DELETE /identity/v3/projects/91560859b3fc4c33a5bb63ae2b99b8eb => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:25.377737 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37559-17b7-4624-995a-0c76ddfd3823 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.378153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37559-17b7-4624-995a-0c76ddfd3823 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.378699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37559-17b7-4624-995a-0c76ddfd3823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.392663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:25.413982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dfc37559-17b7-4624-995a-0c76ddfd3823 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:25.416494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dfc37559-17b7-4624-995a-0c76ddfd3823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:25.505240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1059/2152] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:25.514678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c05fe-6ec5-4dd8-913b-749aa50ed763 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.515369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c05fe-6ec5-4dd8-913b-749aa50ed763 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.515802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c05fe-6ec5-4dd8-913b-749aa50ed763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.542470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-628c05fe-6ec5-4dd8-913b-749aa50ed763 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:25.544161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-628c05fe-6ec5-4dd8-913b-749aa50ed763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:25.557150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1094/2153] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:25.558982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc37559-17b7-4624-995a-0c76ddfd3823 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:25.569582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc37559-17b7-4624-995a-0c76ddfd3823 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:25.575752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:25.576621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.576830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.577133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.578347 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:25.578502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:25.580176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:25.598032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1060/2154] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:36:25.599803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c05fe-6ec5-4dd8-913b-749aa50ed763 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:25.609703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c05fe-6ec5-4dd8-913b-749aa50ed763 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:25.615523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaf710-90b8-4703-a5e1-9f9d40832f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:25.616128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf710-90b8-4703-a5e1-9f9d40832f7e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.616429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf710-90b8-4703-a5e1-9f9d40832f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.616698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf710-90b8-4703-a5e1-9f9d40832f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.617427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaf710-90b8-4703-a5e1-9f9d40832f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:25.619843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaf710-90b8-4703-a5e1-9f9d40832f7e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:25.655693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1095/2155] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:25 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:25.662764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:25.685820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d17b5-7e53-4eb5-8477-5ed83b22e939 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:25.786338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968902-40a9-448b-894e-dd5885324a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:25.786981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968902-40a9-448b-894e-dd5885324a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.787257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968902-40a9-448b-894e-dd5885324a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.787526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968902-40a9-448b-894e-dd5885324a48 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.788170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968902-40a9-448b-894e-dd5885324a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:25.790071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968902-40a9-448b-894e-dd5885324a48 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:25.822989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1061/2156] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:25 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:25.825927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaf710-90b8-4703-a5e1-9f9d40832f7e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:25.843153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaf710-90b8-4703-a5e1-9f9d40832f7e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:25.850138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1578398-b17d-4368-b854-63f39a515cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:25.850843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1578398-b17d-4368-b854-63f39a515cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.851098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1578398-b17d-4368-b854-63f39a515cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.851372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1578398-b17d-4368-b854-63f39a515cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.852020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1578398-b17d-4368-b854-63f39a515cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:25.854579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1578398-b17d-4368-b854-63f39a515cc9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:25.870823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1096/2157] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:25.871721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968902-40a9-448b-894e-dd5885324a48 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:25.882750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968902-40a9-448b-894e-dd5885324a48 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:25.888003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb34ade-332c-484b-b959-17947de3bf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:25.888620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34ade-332c-484b-b959-17947de3bf2f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.888904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34ade-332c-484b-b959-17947de3bf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.889174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34ade-332c-484b-b959-17947de3bf2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.889831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb34ade-332c-484b-b959-17947de3bf2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:25.891750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb34ade-332c-484b-b959-17947de3bf2f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:25.914980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1062/2158] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:25.923775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1578398-b17d-4368-b854-63f39a515cc9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:25.935171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1578398-b17d-4368-b854-63f39a515cc9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:25.941384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d062f0-5ba2-499e-919c-19213bdc6849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:25.942134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d062f0-5ba2-499e-919c-19213bdc6849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:25.942467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d062f0-5ba2-499e-919c-19213bdc6849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:25.942853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d062f0-5ba2-499e-919c-19213bdc6849 admin admin] PATH_INFO: `/v3/projects/ce792022f0e14132975e618c57adff10/users/20a19b66624647599412e19e8537dcba/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:25.974313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d062f0-5ba2-499e-919c-19213bdc6849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce792022f0e14132975e618c57adff10, user_id=20a19b66624647599412e19e8537dcba, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:25.984384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d062f0-5ba2-499e-919c-19213bdc6849 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.012688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1097/2159] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:25 2022] PUT /identity/v3/projects/ce792022f0e14132975e618c57adff10/users/20a19b66624647599412e19e8537dcba/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:26.023792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6165a-1506-4014-8e3b-33c2ae13caf3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.024034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6165a-1506-4014-8e3b-33c2ae13caf3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.024307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6165a-1506-4014-8e3b-33c2ae13caf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.055696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d2a6165a-1506-4014-8e3b-33c2ae13caf3 None None] MFA Rules not processed for user `20a19b66624647599412e19e8537dcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:26.056994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d2a6165a-1506-4014-8e3b-33c2ae13caf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:26.149518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:26.160163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1063/2160] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:26.161166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d062f0-5ba2-499e-919c-19213bdc6849 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.228126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6165a-1506-4014-8e3b-33c2ae13caf3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.229911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d2a6165a-1506-4014-8e3b-33c2ae13caf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:26.239867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d062f0-5ba2-499e-919c-19213bdc6849 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:26.249220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6165a-1506-4014-8e3b-33c2ae13caf3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:26.251923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81d7df-0863-476f-b9b0-6fd553ac54e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:26.252723 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81d7df-0863-476f-b9b0-6fd553ac54e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.253052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81d7df-0863-476f-b9b0-6fd553ac54e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.253373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81d7df-0863-476f-b9b0-6fd553ac54e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.253967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81d7df-0863-476f-b9b0-6fd553ac54e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:26.256215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81d7df-0863-476f-b9b0-6fd553ac54e9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.256809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:26.258006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.258518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.259406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] PATH_INFO: `/v3/users/7be7950a96534bdfa9a46cf4938a1c27` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.261643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be7950a96534bdfa9a46cf4938a1c27)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:26.264146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.294453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1098/2161] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:26 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:26.299582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81d7df-0863-476f-b9b0-6fd553ac54e9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.307299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:26.345392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be7950a96534bdfa9a46cf4938a1c27'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:26.355653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be7950a96534bdfa9a46cf4938a1c27'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:26.356847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be7950a96534bdfa9a46cf4938a1c27'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:26.375945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be7950a96534bdfa9a46cf4938a1c27'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:26.376769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd257a18-a188-4fdf-97e5-d981d335d5d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be7950a96534bdfa9a46cf4938a1c27'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:26.389724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1064/2162] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:26 2022] DELETE /identity/v3/users/7be7950a96534bdfa9a46cf4938a1c27 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:26.394240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860d45-04c8-4a72-836b-18146e2407b2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.394471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860d45-04c8-4a72-836b-18146e2407b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.394771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860d45-04c8-4a72-836b-18146e2407b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.418563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bd860d45-04c8-4a72-836b-18146e2407b2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:26.419134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81d7df-0863-476f-b9b0-6fd553ac54e9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:26.419841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd860d45-04c8-4a72-836b-18146e2407b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:26.495907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1065/2163] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:26.502407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:26.503180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.503499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.503752 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.504433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:26.506551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.506993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd860d45-04c8-4a72-836b-18146e2407b2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.508583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd860d45-04c8-4a72-836b-18146e2407b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:26.529856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1099/2164] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:26 2022] POST /identity/v3/users => generated 296 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:26.534389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.536026 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:26.547594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42277a9-6d89-41fd-ab11-a4d1bbb16ee5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:26.552210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdce36-b40b-4da1-8c6c-65b4b19364ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:26.552819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdce36-b40b-4da1-8c6c-65b4b19364ba admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.553098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdce36-b40b-4da1-8c6c-65b4b19364ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.553374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdce36-b40b-4da1-8c6c-65b4b19364ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.554024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdce36-b40b-4da1-8c6c-65b4b19364ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:26.555673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdce36-b40b-4da1-8c6c-65b4b19364ba admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.565810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1100/2165] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:26.567507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd860d45-04c8-4a72-836b-18146e2407b2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:26.570019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdce36-b40b-4da1-8c6c-65b4b19364ba admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.571079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2f5b3-759e-4a89-8bf9-36d1fc83ba27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:26.571689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2f5b3-759e-4a89-8bf9-36d1fc83ba27 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.571962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2f5b3-759e-4a89-8bf9-36d1fc83ba27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.572227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2f5b3-759e-4a89-8bf9-36d1fc83ba27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.580346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdce36-b40b-4da1-8c6c-65b4b19364ba admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:26.582168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2f5b3-759e-4a89-8bf9-36d1fc83ba27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:26.583754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2f5b3-759e-4a89-8bf9-36d1fc83ba27 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.585333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d28859-5bb4-4300-a274-3c9eada5340a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:26.585979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d28859-5bb4-4300-a274-3c9eada5340a admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.586290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d28859-5bb4-4300-a274-3c9eada5340a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.586533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d28859-5bb4-4300-a274-3c9eada5340a admin admin] PATH_INFO: `/v3/projects/316c2c96675d487ca1e35bc65b447652/users/5b5f20a575d44327bc1d09e49ef3644d/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.593420 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1066/2166] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:36:26.602452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d28859-5bb4-4300-a274-3c9eada5340a admin admin] RBAC: Authorizing `identity:create_grant(project_id=316c2c96675d487ca1e35bc65b447652, user_id=5b5f20a575d44327bc1d09e49ef3644d, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:26.605147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d28859-5bb4-4300-a274-3c9eada5340a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.619784 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1101/2167] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:26 2022] PUT /identity/v3/projects/316c2c96675d487ca1e35bc65b447652/users/5b5f20a575d44327bc1d09e49ef3644d/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:26.624112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2f5b3-759e-4a89-8bf9-36d1fc83ba27 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.634621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2f5b3-759e-4a89-8bf9-36d1fc83ba27 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:26.654997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:26.711793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e03bcc-7584-4b86-ba00-75e5e483c59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:26.712702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e03bcc-7584-4b86-ba00-75e5e483c59e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.712991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e03bcc-7584-4b86-ba00-75e5e483c59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.713262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e03bcc-7584-4b86-ba00-75e5e483c59e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.713921 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e03bcc-7584-4b86-ba00-75e5e483c59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:26.715572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e03bcc-7584-4b86-ba00-75e5e483c59e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.726969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1067/2168] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:26.731606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d28859-5bb4-4300-a274-3c9eada5340a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.750856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d28859-5bb4-4300-a274-3c9eada5340a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:26.756024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:26.756706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.756988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.757267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] PATH_INFO: `/v3/projects/316c2c96675d487ca1e35bc65b447652/users/5b5f20a575d44327bc1d09e49ef3644d/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.759375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=316c2c96675d487ca1e35bc65b447652, user_id=5b5f20a575d44327bc1d09e49ef3644d, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:26.762129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:26.773846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1102/2169] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:26 2022] PUT /identity/v3/projects/316c2c96675d487ca1e35bc65b447652/users/5b5f20a575d44327bc1d09e49ef3644d/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:26.785607 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840d2c1-bc11-4a77-b189-70240afe939c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:26.786059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840d2c1-bc11-4a77-b189-70240afe939c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:26.786369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840d2c1-bc11-4a77-b189-70240afe939c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:26.831973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b840d2c1-bc11-4a77-b189-70240afe939c None None] MFA Rules not processed for user `5b5f20a575d44327bc1d09e49ef3644d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:26.833699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b840d2c1-bc11-4a77-b189-70240afe939c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:26.918451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1068/2170] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:26.931839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.934121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:26.992845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840d2c1-bc11-4a77-b189-70240afe939c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:26.997352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b840d2c1-bc11-4a77-b189-70240afe939c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:27.094482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738ed53-14ce-4f8d-8cdd-05ab13428e3e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:27.107073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0636c6-7a97-407c-bb0c-64e135ed9945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:27.107946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0636c6-7a97-407c-bb0c-64e135ed9945 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:27.108416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0636c6-7a97-407c-bb0c-64e135ed9945 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:27.108619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0636c6-7a97-407c-bb0c-64e135ed9945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:27.118180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:27.124381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0636c6-7a97-407c-bb0c-64e135ed9945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:27.126234 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0636c6-7a97-407c-bb0c-64e135ed9945 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:27.139597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1103/2171] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:27.140230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840d2c1-bc11-4a77-b189-70240afe939c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:27.144713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:27.145351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:27.145628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:27.145893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] PATH_INFO: `/v3/projects/164d7a0a14fd483791eeb4acf65f52e5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:27.147149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=164d7a0a14fd483791eeb4acf65f52e5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:27.149166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:27.195886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:27.205308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164d7a0a14fd483791eeb4acf65f52e5', 'request_id': 'req-d930b1be-1495-4b7c-9393-9c7726e02763'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.213414 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164d7a0a14fd483791eeb4acf65f52e5', 'request_id': 'req-d930b1be-1495-4b7c-9393-9c7726e02763'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.223763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164d7a0a14fd483791eeb4acf65f52e5', 'request_id': 'req-d930b1be-1495-4b7c-9393-9c7726e02763'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.244260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] The token cache is being invalidate because project 164d7a0a14fd483791eeb4acf65f52e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:27.244761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.245841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.249369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1069/2172] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:26 2022] DELETE /identity/v3/projects/164d7a0a14fd483791eeb4acf65f52e5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:27.252935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0636c6-7a97-407c-bb0c-64e135ed9945 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:27.257027 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:27.258436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:27.268906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0636c6-7a97-407c-bb0c-64e135ed9945 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:27.360715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930b1be-1495-4b7c-9393-9c7726e02763 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:27.367086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:27.367750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:27.368373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:27.369498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:27.375701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:27.378331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:27.379110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:27.379440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:27.382069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] PATH_INFO: `/v3/users/50e295110ad643c8a3dbfb4ec1e21548` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:27.382069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50e295110ad643c8a3dbfb4ec1e21548)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:27.383094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:27.458356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:27.459781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:27.465597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50e295110ad643c8a3dbfb4ec1e21548'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.470577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1070/2173] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:36:27.476286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50e295110ad643c8a3dbfb4ec1e21548'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.477464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50e295110ad643c8a3dbfb4ec1e21548'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.508324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50e295110ad643c8a3dbfb4ec1e21548'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.509348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1236f50-c9d1-4379-9e1e-9fca2345bc85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50e295110ad643c8a3dbfb4ec1e21548'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.526602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1104/2174] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:27 2022] DELETE /identity/v3/users/50e295110ad643c8a3dbfb4ec1e21548 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:27.848101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:27.849866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:27.865859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326277-56a2-4fda-b4bd-d6852367efab service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:27.927365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:27.928012 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:27.928278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:27.928555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] PATH_INFO: `/v3/projects/79bb0cc9213247e6a470a86676aee4c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:27.929758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bb0cc9213247e6a470a86676aee4c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:27.931740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:27.957284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:27.981591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bb0cc9213247e6a470a86676aee4c1', 'request_id': 'req-d37389ec-dea7-414a-af59-8f48936b3cc0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.987808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bb0cc9213247e6a470a86676aee4c1', 'request_id': 'req-d37389ec-dea7-414a-af59-8f48936b3cc0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:27.996542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bb0cc9213247e6a470a86676aee4c1', 'request_id': 'req-d37389ec-dea7-414a-af59-8f48936b3cc0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:28.008473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] The token cache is being invalidate because project 79bb0cc9213247e6a470a86676aee4c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:28.008688 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:28.009342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d37389ec-dea7-414a-af59-8f48936b3cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:28.010981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1071/2175] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:27 2022] DELETE /identity/v3/projects/79bb0cc9213247e6a470a86676aee4c1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:28.016364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b858d5-88f3-43ef-8870-a17acd9f35d3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.016656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b858d5-88f3-43ef-8870-a17acd9f35d3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.016932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b858d5-88f3-43ef-8870-a17acd9f35d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.037720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f5b858d5-88f3-43ef-8870-a17acd9f35d3 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:28.038973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f5b858d5-88f3-43ef-8870-a17acd9f35d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:28.101643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1105/2176] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:28.109475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd807b-5e11-4d29-8519-6624516f35db None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.109855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd807b-5e11-4d29-8519-6624516f35db None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.110084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd807b-5e11-4d29-8519-6624516f35db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.130792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e5dd807b-5e11-4d29-8519-6624516f35db None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:28.131996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e5dd807b-5e11-4d29-8519-6624516f35db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:28.140429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1072/2177] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:28.144731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b858d5-88f3-43ef-8870-a17acd9f35d3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.154870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b858d5-88f3-43ef-8870-a17acd9f35d3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.160301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.160903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.161176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.161435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.162086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:28.162381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.164024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.172879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:28.176814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1106/2178] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:28.181743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd807b-5e11-4d29-8519-6624516f35db None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.191175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd807b-5e11-4d29-8519-6624516f35db None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.195887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085f830-c485-4efa-9393-e2cb1beb9580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.196503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f830-c485-4efa-9393-e2cb1beb9580 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.196814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f830-c485-4efa-9393-e2cb1beb9580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.197078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f830-c485-4efa-9393-e2cb1beb9580 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.197709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085f830-c485-4efa-9393-e2cb1beb9580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.199638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085f830-c485-4efa-9393-e2cb1beb9580 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.229046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1073/2179] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:28 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:28.233333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.243902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d5d41-55f8-4192-9bae-00a530f390a3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.303398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0d1c3-7a0b-40ba-9649-2f40c905df33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.304382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0d1c3-7a0b-40ba-9649-2f40c905df33 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.304684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0d1c3-7a0b-40ba-9649-2f40c905df33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.305155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0d1c3-7a0b-40ba-9649-2f40c905df33 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.306268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0d1c3-7a0b-40ba-9649-2f40c905df33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.309516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0d1c3-7a0b-40ba-9649-2f40c905df33 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.334954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1107/2180] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:28 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:28.339853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085f830-c485-4efa-9393-e2cb1beb9580 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.352001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085f830-c485-4efa-9393-e2cb1beb9580 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.357211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04469-4120-45ab-be7a-5e46378501f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.357809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04469-4120-45ab-be7a-5e46378501f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.358084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04469-4120-45ab-be7a-5e46378501f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.358326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04469-4120-45ab-be7a-5e46378501f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.359006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed04469-4120-45ab-be7a-5e46378501f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.360728 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed04469-4120-45ab-be7a-5e46378501f4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.371276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1074/2181] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:28.375395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0d1c3-7a0b-40ba-9649-2f40c905df33 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.385807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0d1c3-7a0b-40ba-9649-2f40c905df33 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.390978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a13d3e-66bf-4813-b653-c3db8ba430cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.391604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a13d3e-66bf-4813-b653-c3db8ba430cf admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.391874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a13d3e-66bf-4813-b653-c3db8ba430cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.392168 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a13d3e-66bf-4813-b653-c3db8ba430cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.392806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a13d3e-66bf-4813-b653-c3db8ba430cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.394451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a13d3e-66bf-4813-b653-c3db8ba430cf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.405471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1108/2182] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:28.410524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04469-4120-45ab-be7a-5e46378501f4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.421155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04469-4120-45ab-be7a-5e46378501f4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.426383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420757e-26ad-45bd-93d1-afcc7b884825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.427091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420757e-26ad-45bd-93d1-afcc7b884825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.427368 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420757e-26ad-45bd-93d1-afcc7b884825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.427638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420757e-26ad-45bd-93d1-afcc7b884825 admin admin] PATH_INFO: `/v3/projects/204277b4b2e1499aa8469cee50d3fd1e/users/939d1bfdccee4339b0c7378831f061ee/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.444427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420757e-26ad-45bd-93d1-afcc7b884825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=204277b4b2e1499aa8469cee50d3fd1e, user_id=939d1bfdccee4339b0c7378831f061ee, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.447167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420757e-26ad-45bd-93d1-afcc7b884825 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.462546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1075/2183] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:28 2022] PUT /identity/v3/projects/204277b4b2e1499aa8469cee50d3fd1e/users/939d1bfdccee4339b0c7378831f061ee/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:28.475329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553ebe7-5b3b-484c-84c6-f84c511f8407 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.475663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553ebe7-5b3b-484c-84c6-f84c511f8407 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.475926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553ebe7-5b3b-484c-84c6-f84c511f8407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.507721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1553ebe7-5b3b-484c-84c6-f84c511f8407 None None] MFA Rules not processed for user `939d1bfdccee4339b0c7378831f061ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:28.508939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1553ebe7-5b3b-484c-84c6-f84c511f8407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:28.579279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1109/2184] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:28.583660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420757e-26ad-45bd-93d1-afcc7b884825 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.654486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420757e-26ad-45bd-93d1-afcc7b884825 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.660094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711be2f-b4ea-4764-ac3d-0feea041fe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.660699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711be2f-b4ea-4764-ac3d-0feea041fe66 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.660965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711be2f-b4ea-4764-ac3d-0feea041fe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.661222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711be2f-b4ea-4764-ac3d-0feea041fe66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.661876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711be2f-b4ea-4764-ac3d-0feea041fe66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.663895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711be2f-b4ea-4764-ac3d-0feea041fe66 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.692660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1076/2185] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:28 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:28.696688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553ebe7-5b3b-484c-84c6-f84c511f8407 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.707730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553ebe7-5b3b-484c-84c6-f84c511f8407 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.777458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f2e7-6d11-4279-95d5-5656398c7070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.778079 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f2e7-6d11-4279-95d5-5656398c7070 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.778356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f2e7-6d11-4279-95d5-5656398c7070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.780398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f2e7-6d11-4279-95d5-5656398c7070 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.781074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26f2e7-6d11-4279-95d5-5656398c7070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.783929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26f2e7-6d11-4279-95d5-5656398c7070 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.807456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:28.815101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1110/2186] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:28 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:28.820153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711be2f-b4ea-4764-ac3d-0feea041fe66 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.831029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711be2f-b4ea-4764-ac3d-0feea041fe66 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.836223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67949441-2653-4e59-bd7a-254f00358b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.836832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67949441-2653-4e59-bd7a-254f00358b5d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.837111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67949441-2653-4e59-bd7a-254f00358b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.837375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67949441-2653-4e59-bd7a-254f00358b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.838024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67949441-2653-4e59-bd7a-254f00358b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.839655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67949441-2653-4e59-bd7a-254f00358b5d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.851452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1077/2187] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:28.855966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f2e7-6d11-4279-95d5-5656398c7070 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.866660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f2e7-6d11-4279-95d5-5656398c7070 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.872044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ee1f0-7c84-4999-b3b0-5a0e0c1c1a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.872711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ee1f0-7c84-4999-b3b0-5a0e0c1c1a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.872984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ee1f0-7c84-4999-b3b0-5a0e0c1c1a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.873253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ee1f0-7c84-4999-b3b0-5a0e0c1c1a3f admin admin] PATH_INFO: `/v3/projects/1706a36d42bd4a0aab84ed39cfa4d551/users/669e02fe17714630b044eac42b3fd151/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.889342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ee1f0-7c84-4999-b3b0-5a0e0c1c1a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1706a36d42bd4a0aab84ed39cfa4d551, user_id=669e02fe17714630b044eac42b3fd151, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.892143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ee1f0-7c84-4999-b3b0-5a0e0c1c1a3f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.906808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1111/2188] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:28 2022] PUT /identity/v3/projects/1706a36d42bd4a0aab84ed39cfa4d551/users/669e02fe17714630b044eac42b3fd151/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:28.910945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67949441-2653-4e59-bd7a-254f00358b5d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:28.921462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67949441-2653-4e59-bd7a-254f00358b5d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:28.942238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:28.982625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6a8ff-2adc-4113-a2a0-27d07ec0d521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:28.983264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a8ff-2adc-4113-a2a0-27d07ec0d521 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:28.983540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a8ff-2adc-4113-a2a0-27d07ec0d521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:28.983907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a8ff-2adc-4113-a2a0-27d07ec0d521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:28.984948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6a8ff-2adc-4113-a2a0-27d07ec0d521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:28.986539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6a8ff-2adc-4113-a2a0-27d07ec0d521 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:28.996342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1078/2189] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:29.000961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ee1f0-7c84-4999-b3b0-5a0e0c1c1a3f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:29.013212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ee1f0-7c84-4999-b3b0-5a0e0c1c1a3f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:29.020771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:29.021746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:29.022107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:29.022376 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] PATH_INFO: `/v3/projects/1706a36d42bd4a0aab84ed39cfa4d551/users/669e02fe17714630b044eac42b3fd151/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:29.025169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1706a36d42bd4a0aab84ed39cfa4d551, user_id=669e02fe17714630b044eac42b3fd151, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:29.029332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:29.040197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1112/2190] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:28 2022] PUT /identity/v3/projects/1706a36d42bd4a0aab84ed39cfa4d551/users/669e02fe17714630b044eac42b3fd151/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:29.050598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eec59-67b1-43aa-9bea-4b6defbbc655 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:29.050917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eec59-67b1-43aa-9bea-4b6defbbc655 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:29.051221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81eec59-67b1-43aa-9bea-4b6defbbc655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:29.082887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c81eec59-67b1-43aa-9bea-4b6defbbc655 None None] MFA Rules not processed for user `669e02fe17714630b044eac42b3fd151`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:29.084047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c81eec59-67b1-43aa-9bea-4b6defbbc655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:29.166975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1079/2191] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:29.180594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:29.182529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:29.252119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cff675-2cc0-413f-947c-399d2aa2fd2b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:29.256386 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:29.256994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:29.257270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:29.257542 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:29.267275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:29.268709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:29.277971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1113/2192] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:29.372134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81eec59-67b1-43aa-9bea-4b6defbbc655 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:29.382676 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81eec59-67b1-43aa-9bea-4b6defbbc655 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:29.386937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742674a-cf49-4c7f-86a2-c54e467d2dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:29.387541 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742674a-cf49-4c7f-86a2-c54e467d2dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:29.387813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742674a-cf49-4c7f-86a2-c54e467d2dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:29.388075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742674a-cf49-4c7f-86a2-c54e467d2dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:29.448613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742674a-cf49-4c7f-86a2-c54e467d2dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:29.450059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742674a-cf49-4c7f-86a2-c54e467d2dd4 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:29.459477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1080/2193] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:32.252685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:32.254339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:32.326782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1f263-5784-4581-8cc5-8517775e0d15 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:32.334228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:32.334847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:32.335122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:32.335390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] PATH_INFO: `/v3/users/20a19b66624647599412e19e8537dcba` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:32.336560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a19b66624647599412e19e8537dcba)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:32.338597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:32.390639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a19b66624647599412e19e8537dcba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:32.399958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a19b66624647599412e19e8537dcba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:32.400582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a19b66624647599412e19e8537dcba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:32.409076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:32.414786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a19b66624647599412e19e8537dcba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:32.415567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a19b66624647599412e19e8537dcba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:32.430594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1114/2194] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:32 2022] DELETE /identity/v3/users/20a19b66624647599412e19e8537dcba => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:32.435252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffe2d-d401-48c1-9ff5-52d3f30c94b7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:32.435757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffe2d-d401-48c1-9ff5-52d3f30c94b7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:32.436061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffe2d-d401-48c1-9ff5-52d3f30c94b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:32.457220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-971ffe2d-d401-48c1-9ff5-52d3f30c94b7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:32.458432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-971ffe2d-d401-48c1-9ff5-52d3f30c94b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:32.498431 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:32.520899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1081/2195] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:32.531750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:32.533494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:32.602963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e521289-6d37-4ff4-8c67-92fd94870fa1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:32.607422 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe248b8-2295-4b20-a724-f6f889393cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:32.608049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe248b8-2295-4b20-a724-f6f889393cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:32.608323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe248b8-2295-4b20-a724-f6f889393cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:32.608607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe248b8-2295-4b20-a724-f6f889393cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:32.619056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe248b8-2295-4b20-a724-f6f889393cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:32.620496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe248b8-2295-4b20-a724-f6f889393cb2 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:32.630846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1115/2196] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:36:32.910355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ffe2d-d401-48c1-9ff5-52d3f30c94b7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:32.912286 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-971ffe2d-d401-48c1-9ff5-52d3f30c94b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:32.928320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ffe2d-d401-48c1-9ff5-52d3f30c94b7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:32.934296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:32.934850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:32.935251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:32.935549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] PATH_INFO: `/v3/projects/ce792022f0e14132975e618c57adff10` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:32.936891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce792022f0e14132975e618c57adff10)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:32.939397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:32.990198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce792022f0e14132975e618c57adff10', 'request_id': 'req-4e1d60ea-128b-4d62-9e82-70487cded468'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:32.996505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce792022f0e14132975e618c57adff10', 'request_id': 'req-4e1d60ea-128b-4d62-9e82-70487cded468'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.006026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce792022f0e14132975e618c57adff10', 'request_id': 'req-4e1d60ea-128b-4d62-9e82-70487cded468'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.017208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] The token cache is being invalidate because project ce792022f0e14132975e618c57adff10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:33.017489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.018184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.019814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1082/2197] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:32 2022] DELETE /identity/v3/projects/ce792022f0e14132975e618c57adff10 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:33.024246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe248b8-2295-4b20-a724-f6f889393cb2 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:33.034447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe248b8-2295-4b20-a724-f6f889393cb2 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:33.100764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:33.101405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:33.101680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:33.101952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] PATH_INFO: `/v3/users/5b5f20a575d44327bc1d09e49ef3644d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:33.103105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5f20a575d44327bc1d09e49ef3644d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:33.105099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:33.158537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5f20a575d44327bc1d09e49ef3644d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.167546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5f20a575d44327bc1d09e49ef3644d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.168251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5f20a575d44327bc1d09e49ef3644d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.189185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5f20a575d44327bc1d09e49ef3644d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.189185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ba5384ef-8d8e-4354-97f9-515483bd2b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5f20a575d44327bc1d09e49ef3644d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.193572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1116/2198] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:33 2022] DELETE /identity/v3/users/5b5f20a575d44327bc1d09e49ef3644d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:33.563436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:33.565184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:33.578830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d60ea-128b-4d62-9e82-70487cded468 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:33.677204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:33.677952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:33.678333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:33.678702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] PATH_INFO: `/v3/projects/316c2c96675d487ca1e35bc65b447652` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:33.680321 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=316c2c96675d487ca1e35bc65b447652)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:33.682698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:33.753701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316c2c96675d487ca1e35bc65b447652', 'request_id': 'req-739ac9d8-c91b-46c5-97d6-7c1552d47956'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.762123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316c2c96675d487ca1e35bc65b447652', 'request_id': 'req-739ac9d8-c91b-46c5-97d6-7c1552d47956'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.780284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316c2c96675d487ca1e35bc65b447652', 'request_id': 'req-739ac9d8-c91b-46c5-97d6-7c1552d47956'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.787082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:33.794915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] The token cache is being invalidate because project 316c2c96675d487ca1e35bc65b447652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:33.795449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.796546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-739ac9d8-c91b-46c5-97d6-7c1552d47956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:33.798395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1083/2199] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:33 2022] DELETE /identity/v3/projects/316c2c96675d487ca1e35bc65b447652 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:33.803888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5168dcb-033b-46f4-a175-976a5a32355f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:33.804388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5168dcb-033b-46f4-a175-976a5a32355f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:33.804480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5168dcb-033b-46f4-a175-976a5a32355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:33.815435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:33.827552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f5168dcb-033b-46f4-a175-976a5a32355f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:33.830113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f5168dcb-033b-46f4-a175-976a5a32355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:33.899774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1117/2200] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:33.910091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb79603-c588-4664-93a2-ed050e032516 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:33.910429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb79603-c588-4664-93a2-ed050e032516 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:33.910739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb79603-c588-4664-93a2-ed050e032516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:33.933089 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7bb79603-c588-4664-93a2-ed050e032516 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:33.934307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7bb79603-c588-4664-93a2-ed050e032516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:33.943212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1084/2201] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:33.947797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5168dcb-033b-46f4-a175-976a5a32355f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:33.957973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5168dcb-033b-46f4-a175-976a5a32355f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:33.963061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:33.963646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:33.964003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:33.964332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:33.965088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:33.965447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:33.967174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:33.980510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1118/2202] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:36:33.985200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb79603-c588-4664-93a2-ed050e032516 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:33.995041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb79603-c588-4664-93a2-ed050e032516 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.000302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474e95c-987f-4f7e-9451-ca73f7978090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.000979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474e95c-987f-4f7e-9451-ca73f7978090 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.001246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474e95c-987f-4f7e-9451-ca73f7978090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.001505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474e95c-987f-4f7e-9451-ca73f7978090 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.002128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474e95c-987f-4f7e-9451-ca73f7978090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.004097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474e95c-987f-4f7e-9451-ca73f7978090 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.044840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1085/2203] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:33 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:34.050602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.061368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373db472-d6fb-4778-ac16-af68dbfea339 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.126567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e0c5c-7167-43a0-b756-f62204f7dba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.127225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e0c5c-7167-43a0-b756-f62204f7dba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.127518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e0c5c-7167-43a0-b756-f62204f7dba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.127811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e0c5c-7167-43a0-b756-f62204f7dba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.128531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e0c5c-7167-43a0-b756-f62204f7dba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.130608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e0c5c-7167-43a0-b756-f62204f7dba5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.156839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1119/2204] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:34 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:34.158045 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474e95c-987f-4f7e-9451-ca73f7978090 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.169595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474e95c-987f-4f7e-9451-ca73f7978090 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.174429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acdc4f-9bc1-45c1-a3b2-2685aaa262fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.175055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acdc4f-9bc1-45c1-a3b2-2685aaa262fc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.175327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acdc4f-9bc1-45c1-a3b2-2685aaa262fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.175592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acdc4f-9bc1-45c1-a3b2-2685aaa262fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.176236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acdc4f-9bc1-45c1-a3b2-2685aaa262fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.177823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acdc4f-9bc1-45c1-a3b2-2685aaa262fc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.193074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1086/2205] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:34.198053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e0c5c-7167-43a0-b756-f62204f7dba5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.208171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e0c5c-7167-43a0-b756-f62204f7dba5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.212975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae3a55-0ab6-4763-a305-60c4b806f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.213589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae3a55-0ab6-4763-a305-60c4b806f26e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.213900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae3a55-0ab6-4763-a305-60c4b806f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.214360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae3a55-0ab6-4763-a305-60c4b806f26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.214874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae3a55-0ab6-4763-a305-60c4b806f26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.216575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae3a55-0ab6-4763-a305-60c4b806f26e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.233061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1120/2206] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:34.234047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acdc4f-9bc1-45c1-a3b2-2685aaa262fc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.244277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acdc4f-9bc1-45c1-a3b2-2685aaa262fc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.249292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf438-3f2b-427f-88d5-f564d08d27a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.249944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf438-3f2b-427f-88d5-f564d08d27a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.250415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf438-3f2b-427f-88d5-f564d08d27a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.250874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf438-3f2b-427f-88d5-f564d08d27a1 admin admin] PATH_INFO: `/v3/projects/355be16b85494132a51e66eb85f9e289/users/3985b3697fe34fec88253dcbe501a30d/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.267645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bf438-3f2b-427f-88d5-f564d08d27a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=355be16b85494132a51e66eb85f9e289, user_id=3985b3697fe34fec88253dcbe501a30d, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.270331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bf438-3f2b-427f-88d5-f564d08d27a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.285072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1087/2207] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:34 2022] PUT /identity/v3/projects/355be16b85494132a51e66eb85f9e289/users/3985b3697fe34fec88253dcbe501a30d/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:34.295497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6c736-93a1-4423-b11d-d1aacce6bf5b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.295776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6c736-93a1-4423-b11d-d1aacce6bf5b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.296044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6c736-93a1-4423-b11d-d1aacce6bf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.323995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-dcb6c736-93a1-4423-b11d-d1aacce6bf5b None None] MFA Rules not processed for user `3985b3697fe34fec88253dcbe501a30d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:34.326104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dcb6c736-93a1-4423-b11d-d1aacce6bf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:34.395295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:34.400741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1121/2208] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:34.405993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf438-3f2b-427f-88d5-f564d08d27a1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.477367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf438-3f2b-427f-88d5-f564d08d27a1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.482483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9f879-b654-480e-b965-85772d379cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.483128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9f879-b654-480e-b965-85772d379cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.483387 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9f879-b654-480e-b965-85772d379cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.483646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9f879-b654-480e-b965-85772d379cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.484295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9f879-b654-480e-b965-85772d379cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.486407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9f879-b654-480e-b965-85772d379cbf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.513959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1088/2209] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:34 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:34.518338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6c736-93a1-4423-b11d-d1aacce6bf5b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.530576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6c736-93a1-4423-b11d-d1aacce6bf5b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.593984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844323-dcbe-420e-91a6-d751f66ef3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.594595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844323-dcbe-420e-91a6-d751f66ef3ef admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.594911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844323-dcbe-420e-91a6-d751f66ef3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.595180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844323-dcbe-420e-91a6-d751f66ef3ef admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.595831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8844323-dcbe-420e-91a6-d751f66ef3ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.598214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8844323-dcbe-420e-91a6-d751f66ef3ef admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.622775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1122/2210] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:34 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:34.627174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9f879-b654-480e-b965-85772d379cbf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.638209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9f879-b654-480e-b965-85772d379cbf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.644597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3182f-b2ab-4dfd-8b40-13d5e75e5cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.644744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3182f-b2ab-4dfd-8b40-13d5e75e5cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.645018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3182f-b2ab-4dfd-8b40-13d5e75e5cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.645206 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3182f-b2ab-4dfd-8b40-13d5e75e5cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.645923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3182f-b2ab-4dfd-8b40-13d5e75e5cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.647701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3182f-b2ab-4dfd-8b40-13d5e75e5cbc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.661989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1089/2211] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:34.663419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844323-dcbe-420e-91a6-d751f66ef3ef admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.673733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844323-dcbe-420e-91a6-d751f66ef3ef admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.678462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffcc28-fde8-4c8e-8a09-682e2cfe0a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.679177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffcc28-fde8-4c8e-8a09-682e2cfe0a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.679453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffcc28-fde8-4c8e-8a09-682e2cfe0a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.679735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffcc28-fde8-4c8e-8a09-682e2cfe0a61 admin admin] PATH_INFO: `/v3/projects/18908b8cbe2145af90ee59a80999143c/users/47d57f3cd675446cac58b8e3ac464057/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.706994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffcc28-fde8-4c8e-8a09-682e2cfe0a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18908b8cbe2145af90ee59a80999143c, user_id=47d57f3cd675446cac58b8e3ac464057, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.709726 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffcc28-fde8-4c8e-8a09-682e2cfe0a61 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.731323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1123/2212] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:34 2022] PUT /identity/v3/projects/18908b8cbe2145af90ee59a80999143c/users/47d57f3cd675446cac58b8e3ac464057/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:34.735702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3182f-b2ab-4dfd-8b40-13d5e75e5cbc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.747022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3182f-b2ab-4dfd-8b40-13d5e75e5cbc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.809041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b28b20-ada7-4167-9187-3934beaf6983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.809385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28b20-ada7-4167-9187-3934beaf6983 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.809660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28b20-ada7-4167-9187-3934beaf6983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.809931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b28b20-ada7-4167-9187-3934beaf6983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.810575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b28b20-ada7-4167-9187-3934beaf6983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.812293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b28b20-ada7-4167-9187-3934beaf6983 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.829781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1090/2213] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:34.831014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffcc28-fde8-4c8e-8a09-682e2cfe0a61 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.846216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffcc28-fde8-4c8e-8a09-682e2cfe0a61 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:34.851772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:34.852447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.852725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.852995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] PATH_INFO: `/v3/projects/18908b8cbe2145af90ee59a80999143c/users/47d57f3cd675446cac58b8e3ac464057/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.854981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=18908b8cbe2145af90ee59a80999143c, user_id=47d57f3cd675446cac58b8e3ac464057, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:34.857676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:34.868897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1124/2214] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:34 2022] PUT /identity/v3/projects/18908b8cbe2145af90ee59a80999143c/users/47d57f3cd675446cac58b8e3ac464057/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:34.880661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2176-d097-42bd-abc5-ce705ee7453b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:34.881082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2176-d097-42bd-abc5-ce705ee7453b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:34.881359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2176-d097-42bd-abc5-ce705ee7453b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:34.891421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:34.915330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-320a2176-d097-42bd-abc5-ce705ee7453b None None] MFA Rules not processed for user `47d57f3cd675446cac58b8e3ac464057`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:34.916557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-320a2176-d097-42bd-abc5-ce705ee7453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:34.984675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1091/2215] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:34.995965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:34.998068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:35.059638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232382e-5274-4b6c-9fb6-7552bbbee6ea admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:35.063553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d140e-fe82-40c3-b075-5d35119ca287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:35.064239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d140e-fe82-40c3-b075-5d35119ca287 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:35.064578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d140e-fe82-40c3-b075-5d35119ca287 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:35.064913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d140e-fe82-40c3-b075-5d35119ca287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:35.127721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d140e-fe82-40c3-b075-5d35119ca287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:35.129818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d140e-fe82-40c3-b075-5d35119ca287 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:35.144007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1125/2216] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:35.463614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a2176-d097-42bd-abc5-ce705ee7453b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:35.465168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-320a2176-d097-42bd-abc5-ce705ee7453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:35.539938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a2176-d097-42bd-abc5-ce705ee7453b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:35.544281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:35.544908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:35.545180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:35.545444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] PATH_INFO: `/v3/users/939d1bfdccee4339b0c7378831f061ee` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:35.546562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=939d1bfdccee4339b0c7378831f061ee)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:35.548586 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:35.599172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939d1bfdccee4339b0c7378831f061ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:35.608252 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939d1bfdccee4339b0c7378831f061ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:35.608962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939d1bfdccee4339b0c7378831f061ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:35.623805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939d1bfdccee4339b0c7378831f061ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:35.624525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939d1bfdccee4339b0c7378831f061ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:35.638857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1092/2217] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:35 2022] DELETE /identity/v3/users/939d1bfdccee4339b0c7378831f061ee => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:35.644001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9fea-90d7-402a-9ca9-b09d9ae75c57 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:35.644286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9fea-90d7-402a-9ca9-b09d9ae75c57 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:35.644593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9fea-90d7-402a-9ca9-b09d9ae75c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:35.661342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:35.669591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ca0d9fea-90d7-402a-9ca9-b09d9ae75c57 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:35.671667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ca0d9fea-90d7-402a-9ca9-b09d9ae75c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:35.736387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1126/2218] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:35.749565 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:35.751459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:35.762599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:35.831918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e98f5-d023-4751-a15d-59d260f5d593 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:35.836303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37703bc9-24ec-49c2-9f5f-bb7b24d81212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:35.837235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37703bc9-24ec-49c2-9f5f-bb7b24d81212 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:35.837235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37703bc9-24ec-49c2-9f5f-bb7b24d81212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:35.837461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37703bc9-24ec-49c2-9f5f-bb7b24d81212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:35.849530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37703bc9-24ec-49c2-9f5f-bb7b24d81212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:35.851515 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37703bc9-24ec-49c2-9f5f-bb7b24d81212 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:35.869304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1093/2219] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:35.935354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d9fea-90d7-402a-9ca9-b09d9ae75c57 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:35.937643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ca0d9fea-90d7-402a-9ca9-b09d9ae75c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:35.957522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d9fea-90d7-402a-9ca9-b09d9ae75c57 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:35.962802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:35.964060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:35.964401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:35.964852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] PATH_INFO: `/v3/users/5b5b6438b4bd49a5b26bbd9d52e6c8f2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:35.966157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5b6438b4bd49a5b26bbd9d52e6c8f2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:35.968217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:36.026512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5b6438b4bd49a5b26bbd9d52e6c8f2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.037160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5b6438b4bd49a5b26bbd9d52e6c8f2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.037160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5b6438b4bd49a5b26bbd9d52e6c8f2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.051470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5b6438b4bd49a5b26bbd9d52e6c8f2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.052708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5b6438b4bd49a5b26bbd9d52e6c8f2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.067442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1127/2220] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:35 2022] DELETE /identity/v3/users/5b5b6438b4bd49a5b26bbd9d52e6c8f2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:36.072066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adbf91-8f7c-4430-ba46-9548b6ddc7ec None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.072448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adbf91-8f7c-4430-ba46-9548b6ddc7ec None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.072959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adbf91-8f7c-4430-ba46-9548b6ddc7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.097819 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-65adbf91-8f7c-4430-ba46-9548b6ddc7ec None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:36.099248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-65adbf91-8f7c-4430-ba46-9548b6ddc7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:36.171487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1094/2221] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:36.181967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:36.183591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:36.250495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adbf91-8f7c-4430-ba46-9548b6ddc7ec None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:36.251938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-65adbf91-8f7c-4430-ba46-9548b6ddc7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:36.266587 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adbf91-8f7c-4430-ba46-9548b6ddc7ec None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:36.269915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e87af-0b8d-40a8-b66b-46d066157d32 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:36.274203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cbb3d-6c6e-4e37-9299-7d8c002995f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:36.274339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:36.274888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.275077 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cbb3d-6c6e-4e37-9299-7d8c002995f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.275411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cbb3d-6c6e-4e37-9299-7d8c002995f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.275741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.275863 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cbb3d-6c6e-4e37-9299-7d8c002995f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.276026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] PATH_INFO: `/v3/projects/204277b4b2e1499aa8469cee50d3fd1e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.277377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=204277b4b2e1499aa8469cee50d3fd1e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:36.279590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:36.286119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cbb3d-6c6e-4e37-9299-7d8c002995f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:36.287683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cbb3d-6c6e-4e37-9299-7d8c002995f2 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:36.299019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1128/2222] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:36.331960 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '204277b4b2e1499aa8469cee50d3fd1e', 'request_id': 'req-43a98c29-5ae5-490f-a08a-920b1ea945c6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.338071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '204277b4b2e1499aa8469cee50d3fd1e', 'request_id': 'req-43a98c29-5ae5-490f-a08a-920b1ea945c6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.346343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '204277b4b2e1499aa8469cee50d3fd1e', 'request_id': 'req-43a98c29-5ae5-490f-a08a-920b1ea945c6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.363279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] The token cache is being invalidate because project 204277b4b2e1499aa8469cee50d3fd1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:36.363756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.364335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.366003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1095/2223] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:36 2022] DELETE /identity/v3/projects/204277b4b2e1499aa8469cee50d3fd1e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:36.370861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cbb3d-6c6e-4e37-9299-7d8c002995f2 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:36.382682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cbb3d-6c6e-4e37-9299-7d8c002995f2 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:36.401978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:36.445558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:36.446086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.446364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.446634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] PATH_INFO: `/v3/users/669e02fe17714630b044eac42b3fd151` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.447835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=669e02fe17714630b044eac42b3fd151)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:36.450403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:36.523548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '669e02fe17714630b044eac42b3fd151'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.532314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '669e02fe17714630b044eac42b3fd151'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.533022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '669e02fe17714630b044eac42b3fd151'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.548811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '669e02fe17714630b044eac42b3fd151'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.549585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1532652a-aaba-4191-a13f-e0d188690073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '669e02fe17714630b044eac42b3fd151'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.564196 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1129/2224] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:36 2022] DELETE /identity/v3/users/669e02fe17714630b044eac42b3fd151 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:36.595596 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:36.597494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:36.611416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98c29-5ae5-490f-a08a-920b1ea945c6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:36.647017 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:36.685489 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:36.686137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.686442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.687884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] PATH_INFO: `/v3/projects/e279a8dd7da242a7a3d15723eb8126b0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.688294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e279a8dd7da242a7a3d15723eb8126b0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:36.690407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:36.768742 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e279a8dd7da242a7a3d15723eb8126b0', 'request_id': 'req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.779848 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e279a8dd7da242a7a3d15723eb8126b0', 'request_id': 'req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.789927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e279a8dd7da242a7a3d15723eb8126b0', 'request_id': 'req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.800468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] The token cache is being invalidate because project e279a8dd7da242a7a3d15723eb8126b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:36.800746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.801369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:36.803264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1096/2225] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:36 2022] DELETE /identity/v3/projects/e279a8dd7da242a7a3d15723eb8126b0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:36.808189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f32674-6f6a-48bc-ab6f-52f22fc68b8b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.808530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f32674-6f6a-48bc-ab6f-52f22fc68b8b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.808883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f32674-6f6a-48bc-ab6f-52f22fc68b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.828382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-28f32674-6f6a-48bc-ab6f-52f22fc68b8b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:36.829678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-28f32674-6f6a-48bc-ab6f-52f22fc68b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:36.882299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:36.883783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:36.894645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1130/2226] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:36.903817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e1274-8555-4759-bb78-ba49c40cf19b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.904281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e1274-8555-4759-bb78-ba49c40cf19b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.907093 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e1274-8555-4759-bb78-ba49c40cf19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.931695 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d04e1274-8555-4759-bb78-ba49c40cf19b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:36.934082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d04e1274-8555-4759-bb78-ba49c40cf19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:36.946900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1131/2227] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:36.949069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e1274-8555-4759-bb78-ba49c40cf19b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:36.959169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f53dc-e4ad-4dd1-8733-731e3fbd7428 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:36.960370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e1274-8555-4759-bb78-ba49c40cf19b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:36.964342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:36.965200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.965317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:36.965469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.965861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] PATH_INFO: `/v3/projects/1706a36d42bd4a0aab84ed39cfa4d551` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.966570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:36.968277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:36.968402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1706a36d42bd4a0aab84ed39cfa4d551)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:36.968502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:36.968502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:36.968502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:36.969151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:36.976757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:36.991414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1132/2228] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:36.996611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.008769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ee8f5-5571-4b47-94e4-37499067732f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.023301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1706a36d42bd4a0aab84ed39cfa4d551', 'request_id': 'req-eed07c80-8166-4baf-a823-012172a6d42c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.029562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:37.030385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1706a36d42bd4a0aab84ed39cfa4d551', 'request_id': 'req-eed07c80-8166-4baf-a823-012172a6d42c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.039411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1706a36d42bd4a0aab84ed39cfa4d551', 'request_id': 'req-eed07c80-8166-4baf-a823-012172a6d42c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.045298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:37.057664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] The token cache is being invalidate because project 1706a36d42bd4a0aab84ed39cfa4d551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:37.058238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.071282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed07c80-8166-4baf-a823-012172a6d42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.073425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1097/2229] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:36 2022] DELETE /identity/v3/projects/1706a36d42bd4a0aab84ed39cfa4d551 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:37.078024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0aec8-1e7d-4367-bd37-eb1e41955388 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.078339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0aec8-1e7d-4367-bd37-eb1e41955388 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.078663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0aec8-1e7d-4367-bd37-eb1e41955388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.093771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.094647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.094989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.095318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.096179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.098614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.102205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b7a0aec8-1e7d-4367-bd37-eb1e41955388 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:37.103498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7a0aec8-1e7d-4367-bd37-eb1e41955388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.112498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1098/2230] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:37.121937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3c282-6085-452d-a30a-cab7ade0dd26 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.122222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3c282-6085-452d-a30a-cab7ade0dd26 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.122495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3c282-6085-452d-a30a-cab7ade0dd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.132330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1133/2231] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:36 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:37.136132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.137796 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.151233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c1e3c282-6085-452d-a30a-cab7ade0dd26 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:37.152435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c1e3c282-6085-452d-a30a-cab7ade0dd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.211454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b1988-c2ea-4bb7-a4ae-8fe7460c6b31 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.216745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd47566-36ef-459e-aa34-49e29c9578a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.217108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1099/2232] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:37.217406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd47566-36ef-459e-aa34-49e29c9578a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.217700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd47566-36ef-459e-aa34-49e29c9578a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.217983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd47566-36ef-459e-aa34-49e29c9578a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.218683 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd47566-36ef-459e-aa34-49e29c9578a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.221099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd47566-36ef-459e-aa34-49e29c9578a7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.223166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3c282-6085-452d-a30a-cab7ade0dd26 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.235223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3c282-6085-452d-a30a-cab7ade0dd26 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.241825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.242543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.242885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.243083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.243980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:37.244095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.245655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.251544 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1134/2233] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:37 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:37.255619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd47566-36ef-459e-aa34-49e29c9578a7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.262774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1100/2234] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:36:37.264264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.267134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd47566-36ef-459e-aa34-49e29c9578a7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.272088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.272776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.273120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.273456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.274112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.274380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bede04-2aba-4fd3-8a20-07fefbfd19f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.275764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.279145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608c84b-70b2-45e1-a961-b52245d7c1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.279736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608c84b-70b2-45e1-a961-b52245d7c1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.280013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608c84b-70b2-45e1-a961-b52245d7c1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.280283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608c84b-70b2-45e1-a961-b52245d7c1f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.280912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608c84b-70b2-45e1-a961-b52245d7c1f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.283152 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608c84b-70b2-45e1-a961-b52245d7c1f2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.286157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1135/2235] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:37.286976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.288639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.301263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f1d33-8de2-4d39-83c3-5b905cabd093 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.306195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.306852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.307134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.307418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] PATH_INFO: `/v3/users/5068ee7196394acfb215418024d647a5` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.308576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5068ee7196394acfb215418024d647a5)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.310506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.312909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1101/2236] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:37 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:37.313747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608c84b-70b2-45e1-a961-b52245d7c1f2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.363183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5068ee7196394acfb215418024d647a5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.372259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5068ee7196394acfb215418024d647a5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.372944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5068ee7196394acfb215418024d647a5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.376280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608c84b-70b2-45e1-a961-b52245d7c1f2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.380892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.381498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.381771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.382039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.382701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.384306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.385768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5068ee7196394acfb215418024d647a5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.386502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5068ee7196394acfb215418024d647a5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:37.394731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:37.395328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1102/2237] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:37.397220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.399156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.405073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1136/2238] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:37 2022] DELETE /identity/v3/users/5068ee7196394acfb215418024d647a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:37.408276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.409684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.414442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7ee1-8d6d-43e3-a95f-26de553eeece admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.478464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:37.489034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4bc27-58fb-4daf-8e7d-83da1d81e775 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.492803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb1ad2-48a6-4637-8c68-00367666ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.493422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1ad2-48a6-4637-8c68-00367666ce25 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.493572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe9dbd-89df-45ac-8731-c43b9547eb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.493682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1ad2-48a6-4637-8c68-00367666ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.493952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1ad2-48a6-4637-8c68-00367666ce25 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.494227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe9dbd-89df-45ac-8731-c43b9547eb34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.494500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe9dbd-89df-45ac-8731-c43b9547eb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.494591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb1ad2-48a6-4637-8c68-00367666ce25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.494798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe9dbd-89df-45ac-8731-c43b9547eb34 admin admin] PATH_INFO: `/v3/projects/80dd3f85755d4deea3fc0192c1cbbc4b/users/294d5d11423c4d2bad05afacdd999995/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.496559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb1ad2-48a6-4637-8c68-00367666ce25 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.511721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe9dbd-89df-45ac-8731-c43b9547eb34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80dd3f85755d4deea3fc0192c1cbbc4b, user_id=294d5d11423c4d2bad05afacdd999995, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.515604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe9dbd-89df-45ac-8731-c43b9547eb34 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.520602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1103/2239] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:37 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:37.522078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87bc33-774d-40cb-aba6-c2733b6c9408 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.522396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87bc33-774d-40cb-aba6-c2733b6c9408 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.522677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87bc33-774d-40cb-aba6-c2733b6c9408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.530019 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1137/2240] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:37 2022] PUT /identity/v3/projects/80dd3f85755d4deea3fc0192c1cbbc4b/users/294d5d11423c4d2bad05afacdd999995/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:37.539214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe9dbd-89df-45ac-8731-c43b9547eb34 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.544227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4d87bc33-774d-40cb-aba6-c2733b6c9408 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:37.545536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d87bc33-774d-40cb-aba6-c2733b6c9408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.607342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe9dbd-89df-45ac-8731-c43b9547eb34 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.611352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1104/2241] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:37.612346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.612927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.613203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.613473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.614144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.615827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6f403-44d7-4939-a896-11514571c38a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.615956 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.616114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6f403-44d7-4939-a896-11514571c38a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.616380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6f403-44d7-4939-a896-11514571c38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.629116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1138/2242] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:37.629984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.632587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.648586 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-31b6f403-44d7-4939-a896-11514571c38a None None] MFA Rules not processed for user `294d5d11423c4d2bad05afacdd999995`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:37.650148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-31b6f403-44d7-4939-a896-11514571c38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.703751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc0337-939e-48cd-9707-fb332b732517 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.708845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.709719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.710215 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.710676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.723016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.724513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.734603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1105/2243] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:37.735725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6f403-44d7-4939-a896-11514571c38a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.749591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1139/2244] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:37.750660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.752071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.763715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6f403-44d7-4939-a896-11514571c38a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.770620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9638-b631-4bed-b86b-ac490d03e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.770620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9638-b631-4bed-b86b-ac490d03e92e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.771008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9638-b631-4bed-b86b-ac490d03e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.771195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9638-b631-4bed-b86b-ac490d03e92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.771975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d9638-b631-4bed-b86b-ac490d03e92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.773477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d9638-b631-4bed-b86b-ac490d03e92e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.776503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:37.786191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1106/2245] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:37.791164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9638-b631-4bed-b86b-ac490d03e92e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.809401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9638-b631-4bed-b86b-ac490d03e92e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.815498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.816275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.816556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.816865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] PATH_INFO: `/v3/projects/6e9228dedd9f400a9a2cd708e565a346/users/69026847c08040c6b835b1d2802b51b3/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.836038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b57af9-5342-414d-879f-80f7cd3eb9bc service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.838696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9228dedd9f400a9a2cd708e565a346, user_id=69026847c08040c6b835b1d2802b51b3, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.841484 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca026df-3272-4f72-a5fd-2d75e4182991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:37.842248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.842540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca026df-3272-4f72-a5fd-2d75e4182991 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.842986 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca026df-3272-4f72-a5fd-2d75e4182991 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.843349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca026df-3272-4f72-a5fd-2d75e4182991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.853495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca026df-3272-4f72-a5fd-2d75e4182991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:37.855308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca026df-3272-4f72-a5fd-2d75e4182991 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:37.858450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1107/2246] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:37 2022] PUT /identity/v3/projects/6e9228dedd9f400a9a2cd708e565a346/users/69026847c08040c6b835b1d2802b51b3/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:37.859871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:37.869040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.877639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1140/2247] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:37.878984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e94e-966f-40bc-b344-42a2f50d9879 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:37.879525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e94e-966f-40bc-b344-42a2f50d9879 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:37.879984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e94e-966f-40bc-b344-42a2f50d9879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:37.922118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5189e94e-966f-40bc-b344-42a2f50d9879 None None] MFA Rules not processed for user `69026847c08040c6b835b1d2802b51b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:37.933996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:37.940263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5189e94e-966f-40bc-b344-42a2f50d9879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:37.992633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697766f-531f-4d75-869a-fe2b80d1b238 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:37.999507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.000250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.000625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.000904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] PATH_INFO: `/v3/users/3985b3697fe34fec88253dcbe501a30d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.002863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] RBAC: Authorizing `identity:delete_user(user_id=3985b3697fe34fec88253dcbe501a30d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.006096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.092667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1141/2248] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:38.104195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3985b3697fe34fec88253dcbe501a30d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.104840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e94e-966f-40bc-b344-42a2f50d9879 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:38.106445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5189e94e-966f-40bc-b344-42a2f50d9879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.118536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3985b3697fe34fec88253dcbe501a30d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.119458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3985b3697fe34fec88253dcbe501a30d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.136836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3985b3697fe34fec88253dcbe501a30d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.139563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7fe7f85e-55a0-4f40-a0cc-ac92e680bfda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3985b3697fe34fec88253dcbe501a30d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.158739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1108/2249] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:37 2022] DELETE /identity/v3/users/3985b3697fe34fec88253dcbe501a30d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:38.163006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb92d9-7ecd-44c8-bd04-9a1e372148fc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.163470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb92d9-7ecd-44c8-bd04-9a1e372148fc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.163801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb92d9-7ecd-44c8-bd04-9a1e372148fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.211788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e94e-966f-40bc-b344-42a2f50d9879 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:38.212525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-36eb92d9-7ecd-44c8-bd04-9a1e372148fc None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:38.216306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-36eb92d9-7ecd-44c8-bd04-9a1e372148fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.296042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.300237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.300524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.301007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.305744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.343016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1109/2250] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:38.344097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb92d9-7ecd-44c8-bd04-9a1e372148fc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:38.350266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-36eb92d9-7ecd-44c8-bd04-9a1e372148fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.351964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:38.375139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb92d9-7ecd-44c8-bd04-9a1e372148fc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:38.387045 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.388260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.388719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.389305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] PATH_INFO: `/v3/projects/2ce8b9ec039b4e24adfff9bed995a945` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.392441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce8b9ec039b4e24adfff9bed995a945)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.397349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.420055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.421279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.443685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1142/2251] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:36:38.445911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:38.447248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.486005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:38.487348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce8b9ec039b4e24adfff9bed995a945', 'request_id': 'req-b315c15d-b75b-4f25-929c-2fdb7609c6f1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.495456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce8b9ec039b4e24adfff9bed995a945', 'request_id': 'req-b315c15d-b75b-4f25-929c-2fdb7609c6f1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.505394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce8b9ec039b4e24adfff9bed995a945', 'request_id': 'req-b315c15d-b75b-4f25-929c-2fdb7609c6f1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.517854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] The token cache is being invalidate because project 2ce8b9ec039b4e24adfff9bed995a945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:38.518144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.518839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.520434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1110/2252] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:38 2022] DELETE /identity/v3/projects/2ce8b9ec039b4e24adfff9bed995a945 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:38.521196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:38.522213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471e4e7-2cbd-42c2-807d-d26494ad7f4a service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:38.522454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.576353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32981a-19cd-49cc-8c05-f16e91a03c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.576353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32981a-19cd-49cc-8c05-f16e91a03c54 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.576353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32981a-19cd-49cc-8c05-f16e91a03c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.576749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32981a-19cd-49cc-8c05-f16e91a03c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.578480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0c32981a-19cd-49cc-8c05-f16e91a03c54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.588847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315c15d-b75b-4f25-929c-2fdb7609c6f1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:38.590826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32981a-19cd-49cc-8c05-f16e91a03c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.592298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32981a-19cd-49cc-8c05-f16e91a03c54 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.598663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.599337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.599604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.599876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] PATH_INFO: `/v3/users/69026847c08040c6b835b1d2802b51b3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.601057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=69026847c08040c6b835b1d2802b51b3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.603355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.604555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1143/2253] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:38.605994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c3961-a0b6-4fa2-abd7-ea50041c4044 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.606279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c3961-a0b6-4fa2-abd7-ea50041c4044 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.606551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c3961-a0b6-4fa2-abd7-ea50041c4044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.630473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-658c3961-a0b6-4fa2-abd7-ea50041c4044 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:38.631806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-658c3961-a0b6-4fa2-abd7-ea50041c4044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.642621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1144/2254] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:38.651578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec7ce-467b-4723-869f-7cb2981dcada None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.652089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec7ce-467b-4723-869f-7cb2981dcada None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.652407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec7ce-467b-4723-869f-7cb2981dcada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.670110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69026847c08040c6b835b1d2802b51b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.674141 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d14ec7ce-467b-4723-869f-7cb2981dcada None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:38.675456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d14ec7ce-467b-4723-869f-7cb2981dcada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.679576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69026847c08040c6b835b1d2802b51b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.680588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69026847c08040c6b835b1d2802b51b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.685734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1145/2255] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:38.691849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ec7ce-467b-4723-869f-7cb2981dcada None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:38.696805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69026847c08040c6b835b1d2802b51b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.697610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cc03604-18dd-483c-a5d7-df9b5a86aa6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69026847c08040c6b835b1d2802b51b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:38.704363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ec7ce-467b-4723-869f-7cb2981dcada None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:38.709622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.711092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1111/2256] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:38 2022] DELETE /identity/v3/users/69026847c08040c6b835b1d2802b51b3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:38.711248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.711515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.711858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.712553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:38.712857 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.714813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.715744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0043d44-7aa6-48e4-9405-4624b2c018fa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.716030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0043d44-7aa6-48e4-9405-4624b2c018fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.716300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0043d44-7aa6-48e4-9405-4624b2c018fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.729005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1146/2257] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:38.734043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:38.736137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.740487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e0043d44-7aa6-48e4-9405-4624b2c018fa None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:38.741980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0043d44-7aa6-48e4-9405-4624b2c018fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.750911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36033a16-5842-4a3f-878e-09124ca6b5b4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:38.822025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:38.833182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:38.837971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1112/2258] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:38.843741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d7992-3ba5-4189-80ce-c6875a868ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.845307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d7992-3ba5-4189-80ce-c6875a868ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.845684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d7992-3ba5-4189-80ce-c6875a868ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.846027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d7992-3ba5-4189-80ce-c6875a868ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.846777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d7992-3ba5-4189-80ce-c6875a868ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.848381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0043d44-7aa6-48e4-9405-4624b2c018fa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:38.849771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0043d44-7aa6-48e4-9405-4624b2c018fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:38.852892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d7992-3ba5-4189-80ce-c6875a868ff8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.886004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1147/2259] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:38 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:38.890331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d7992-3ba5-4189-80ce-c6875a868ff8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:38.916592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0043d44-7aa6-48e4-9405-4624b2c018fa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:38.965857 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d7992-3ba5-4189-80ce-c6875a868ff8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:38.971861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.972846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.973114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.973381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.974005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.976065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.987234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db251512-2e9c-422c-8d66-b6ed294f4807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:38.987852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251512-2e9c-422c-8d66-b6ed294f4807 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:38.988128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251512-2e9c-422c-8d66-b6ed294f4807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:38.988395 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251512-2e9c-422c-8d66-b6ed294f4807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:38.997752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db251512-2e9c-422c-8d66-b6ed294f4807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:38.999218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1148/2260] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:38 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:38.999357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db251512-2e9c-422c-8d66-b6ed294f4807 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:38.999886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.001861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.010888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1113/2261] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:39.013762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caa9aa-511f-40d1-956f-fbf039151abd admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.015582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db251512-2e9c-422c-8d66-b6ed294f4807 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.018921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.019559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.019838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.020109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] PATH_INFO: `/v3/projects/355be16b85494132a51e66eb85f9e289` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.021252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] RBAC: Authorizing `identity:delete_project(project_id=355be16b85494132a51e66eb85f9e289)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.023217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.026681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db251512-2e9c-422c-8d66-b6ed294f4807 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.032276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232d342-cc12-4734-8ea8-b309aa5becb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.032878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d342-cc12-4734-8ea8-b309aa5becb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.033207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d342-cc12-4734-8ea8-b309aa5becb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.033467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d342-cc12-4734-8ea8-b309aa5becb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.034157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232d342-cc12-4734-8ea8-b309aa5becb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.036447 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232d342-cc12-4734-8ea8-b309aa5becb7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.051207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1114/2262] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:39.052413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232d342-cc12-4734-8ea8-b309aa5becb7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.075912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232d342-cc12-4734-8ea8-b309aa5becb7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.082970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '355be16b85494132a51e66eb85f9e289', 'request_id': 'req-1552850c-3199-4ccc-8b60-31271a23117b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.095366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '355be16b85494132a51e66eb85f9e289', 'request_id': 'req-1552850c-3199-4ccc-8b60-31271a23117b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.105654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '355be16b85494132a51e66eb85f9e289', 'request_id': 'req-1552850c-3199-4ccc-8b60-31271a23117b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.123232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] The token cache is being invalidate because project 355be16b85494132a51e66eb85f9e289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:39.123945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.124987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.127690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1149/2263] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:38 2022] DELETE /identity/v3/projects/355be16b85494132a51e66eb85f9e289 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:39.132043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.134120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.151923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.152545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.152822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.153090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.154428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.156513 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.168707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1115/2264] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:39.175899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.175899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.189314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c81b-33c9-475e-86d8-4db9dc26be08 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.211816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:39.212779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d19d9-fecc-44ea-b5de-84f00567e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.213579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d19d9-fecc-44ea-b5de-84f00567e473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.213833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d19d9-fecc-44ea-b5de-84f00567e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.214363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d19d9-fecc-44ea-b5de-84f00567e473 admin admin] PATH_INFO: `/v3/projects/5bdf070482b344edbc329657aa0b8a20/users/dbd6e4fc6e0f41d6942a6a1bd885cd59/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.228766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552850c-3199-4ccc-8b60-31271a23117b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.235246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.236359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.236538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.236869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] PATH_INFO: `/v3/users/47d57f3cd675446cac58b8e3ac464057` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.238021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d19d9-fecc-44ea-b5de-84f00567e473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bdf070482b344edbc329657aa0b8a20, user_id=dbd6e4fc6e0f41d6942a6a1bd885cd59, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.238152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d57f3cd675446cac58b8e3ac464057)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.240030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.241243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d19d9-fecc-44ea-b5de-84f00567e473 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.256634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1116/2265] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:39 2022] PUT /identity/v3/projects/5bdf070482b344edbc329657aa0b8a20/users/dbd6e4fc6e0f41d6942a6a1bd885cd59/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:39.268415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30aad-aeb2-446a-8c03-4d786469b8cb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.268748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30aad-aeb2-446a-8c03-4d786469b8cb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.269004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30aad-aeb2-446a-8c03-4d786469b8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.299914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d57f3cd675446cac58b8e3ac464057'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.301313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-88e30aad-aeb2-446a-8c03-4d786469b8cb None None] MFA Rules not processed for user `dbd6e4fc6e0f41d6942a6a1bd885cd59`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:39.302653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88e30aad-aeb2-446a-8c03-4d786469b8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.307114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:39.308935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d57f3cd675446cac58b8e3ac464057'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.313685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d57f3cd675446cac58b8e3ac464057'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.331649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d57f3cd675446cac58b8e3ac464057'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.332750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d57f3cd675446cac58b8e3ac464057'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.349243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1150/2266] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:39 2022] DELETE /identity/v3/users/47d57f3cd675446cac58b8e3ac464057 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:39.353152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.354585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.385812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1117/2267] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:39.397227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e30aad-aeb2-446a-8c03-4d786469b8cb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.398846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88e30aad-aeb2-446a-8c03-4d786469b8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.458637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5cf8d-4dc7-496d-8aef-db523068ad0c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.467914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.468584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.468872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.469204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] PATH_INFO: `/v3/projects/6e9228dedd9f400a9a2cd708e565a346` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.471264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9228dedd9f400a9a2cd708e565a346)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.480002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.487194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e30aad-aeb2-446a-8c03-4d786469b8cb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.549146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092dcc3-bfc2-4a94-9956-c2503737ab9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.549905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092dcc3-bfc2-4a94-9956-c2503737ab9b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.550254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092dcc3-bfc2-4a94-9956-c2503737ab9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.550584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092dcc3-bfc2-4a94-9956-c2503737ab9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.551086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9228dedd9f400a9a2cd708e565a346', 'request_id': 'req-7aba928b-7fcb-43cf-954a-efcd04740891'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.560762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9228dedd9f400a9a2cd708e565a346', 'request_id': 'req-7aba928b-7fcb-43cf-954a-efcd04740891'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.568979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9228dedd9f400a9a2cd708e565a346', 'request_id': 'req-7aba928b-7fcb-43cf-954a-efcd04740891'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.584517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] The token cache is being invalidate because project 6e9228dedd9f400a9a2cd708e565a346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:39.584808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.585725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7aba928b-7fcb-43cf-954a-efcd04740891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.587643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1151/2268] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:39 2022] DELETE /identity/v3/projects/6e9228dedd9f400a9a2cd708e565a346 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:39.589071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b973d8a-3c41-4683-a8ae-9cb49ff3d1fb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.589217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b973d8a-3c41-4683-a8ae-9cb49ff3d1fb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.589493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b973d8a-3c41-4683-a8ae-9cb49ff3d1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.603517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:39.623702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092dcc3-bfc2-4a94-9956-c2503737ab9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.624402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3b973d8a-3c41-4683-a8ae-9cb49ff3d1fb None None] MFA Rules not processed for user `294d5d11423c4d2bad05afacdd999995`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:39.626240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092dcc3-bfc2-4a94-9956-c2503737ab9b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.628720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3b973d8a-3c41-4683-a8ae-9cb49ff3d1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.639144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1118/2269] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:36:39.640610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ffefd-068f-463d-9e9b-e4a384fc3243 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.641015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ffefd-068f-463d-9e9b-e4a384fc3243 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.641375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ffefd-068f-463d-9e9b-e4a384fc3243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.678540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f36ffefd-068f-463d-9e9b-e4a384fc3243 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:39.679840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f36ffefd-068f-463d-9e9b-e4a384fc3243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.691326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:39.732883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1152/2270] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:39.763670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1119/2271] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:39.772550 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd20036-d3f3-4606-88eb-ab7d24f8fd94 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.772685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd20036-d3f3-4606-88eb-ab7d24f8fd94 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.772946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd20036-d3f3-4606-88eb-ab7d24f8fd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.775214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ffefd-068f-463d-9e9b-e4a384fc3243 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.778695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f36ffefd-068f-463d-9e9b-e4a384fc3243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.808302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ffefd-068f-463d-9e9b-e4a384fc3243 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.813499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.814125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.814437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.814661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] PATH_INFO: `/v3/projects/18908b8cbe2145af90ee59a80999143c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.815452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1dd20036-d3f3-4606-88eb-ab7d24f8fd94 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:39.815909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18908b8cbe2145af90ee59a80999143c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.816667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1dd20036-d3f3-4606-88eb-ab7d24f8fd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:39.817906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.850773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1153/2272] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:39.859910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd20036-d3f3-4606-88eb-ab7d24f8fd94 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.880213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd20036-d3f3-4606-88eb-ab7d24f8fd94 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.885574 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.886194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.886466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.886747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.887517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:39.887893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.889573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:39.904969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1154/2273] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:39.910819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:39.913330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18908b8cbe2145af90ee59a80999143c', 'request_id': 'req-5ea5772f-d1e3-4220-a848-9ea869175b81'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.925776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18908b8cbe2145af90ee59a80999143c', 'request_id': 'req-5ea5772f-d1e3-4220-a848-9ea869175b81'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.926244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31511b-48d2-4158-a82f-10969147c273 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:39.947153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18908b8cbe2145af90ee59a80999143c', 'request_id': 'req-5ea5772f-d1e3-4220-a848-9ea869175b81'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.962338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] The token cache is being invalidate because project 18908b8cbe2145af90ee59a80999143c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:39.962621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.963441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ea5772f-d1e3-4220-a848-9ea869175b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:39.971838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1120/2274] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:39 2022] DELETE /identity/v3/projects/18908b8cbe2145af90ee59a80999143c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:39.974605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f427-2f57-4a6b-a6b2-5cc5b9707cf2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.975019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f427-2f57-4a6b-a6b2-5cc5b9707cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.975344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f427-2f57-4a6b-a6b2-5cc5b9707cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.994256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:39.995307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:39.995584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:39.995850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:39.996482 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:39.998433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.015934 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6a92f427-2f57-4a6b-a6b2-5cc5b9707cf2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:40.017766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a92f427-2f57-4a6b-a6b2-5cc5b9707cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:40.028529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1121/2275] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:40.036519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1a5c-aadd-4927-bd88-a1b9b1b9f0de None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.036925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1a5c-aadd-4927-bd88-a1b9b1b9f0de None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.037296 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1a5c-aadd-4927-bd88-a1b9b1b9f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.058500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1155/2276] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:39 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:40.058500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.068645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:40.094559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fe8a1a5c-aadd-4927-bd88-a1b9b1b9f0de None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:40.098905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe8a1a5c-aadd-4927-bd88-a1b9b1b9f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:40.177632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283961d-f70e-4562-9070-3648f8300f58 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.185983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bf873-5168-403a-9dcf-88744d40e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.186866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bf873-5168-403a-9dcf-88744d40e471 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.187205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bf873-5168-403a-9dcf-88744d40e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.187967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bf873-5168-403a-9dcf-88744d40e471 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.190330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27bf873-5168-403a-9dcf-88744d40e471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.195977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27bf873-5168-403a-9dcf-88744d40e471 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.230450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1122/2277] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:40.232348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1a5c-aadd-4927-bd88-a1b9b1b9f0de None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.234766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1156/2278] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:40 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:40.241421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bf873-5168-403a-9dcf-88744d40e471 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.249249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:40.250870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1a5c-aadd-4927-bd88-a1b9b1b9f0de None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.256787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.257470 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.257767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.258038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.258536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27bf873-5168-403a-9dcf-88744d40e471 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.259030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:40.259354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.261143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.265490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a130a-89d4-44f3-9096-20ada42e3e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.266641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a130a-89d4-44f3-9096-20ada42e3e1e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.266965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a130a-89d4-44f3-9096-20ada42e3e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.267233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a130a-89d4-44f3-9096-20ada42e3e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.267884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a130a-89d4-44f3-9096-20ada42e3e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.270057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a130a-89d4-44f3-9096-20ada42e3e1e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.274555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1123/2279] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:40.279540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.284057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1157/2280] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:40.291991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a130a-89d4-44f3-9096-20ada42e3e1e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.301157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6b646-712d-49ec-ab3b-6420ad079cb0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.304177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a130a-89d4-44f3-9096-20ada42e3e1e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.307673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a1367-7134-472f-8bbe-ba1b0bd32471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.308304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a1367-7134-472f-8bbe-ba1b0bd32471 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.308583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a1367-7134-472f-8bbe-ba1b0bd32471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.308855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a1367-7134-472f-8bbe-ba1b0bd32471 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.309146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b4ea0-b816-4c5b-be83-2abfad5ea020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.309495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a1367-7134-472f-8bbe-ba1b0bd32471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.309883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b4ea0-b816-4c5b-be83-2abfad5ea020 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.310162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b4ea0-b816-4c5b-be83-2abfad5ea020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.310454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b4ea0-b816-4c5b-be83-2abfad5ea020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.311608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a1367-7134-472f-8bbe-ba1b0bd32471 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.311930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b4ea0-b816-4c5b-be83-2abfad5ea020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.314116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b4ea0-b816-4c5b-be83-2abfad5ea020 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.330977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1158/2281] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:40.332145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b4ea0-b816-4c5b-be83-2abfad5ea020 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.341966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:40.347783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b4ea0-b816-4c5b-be83-2abfad5ea020 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.353437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066320bd-b8be-499d-8934-fab39b0731ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.354841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066320bd-b8be-499d-8934-fab39b0731ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.355229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066320bd-b8be-499d-8934-fab39b0731ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.355855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066320bd-b8be-499d-8934-fab39b0731ae admin admin] PATH_INFO: `/v3/projects/bdf51e13c4b9456e95bd536552367d8a/users/4c1fd6d99758441486315678845b4673/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.373140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1124/2282] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:40 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:40.374709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a1367-7134-472f-8bbe-ba1b0bd32471 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.378746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066320bd-b8be-499d-8934-fab39b0731ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf51e13c4b9456e95bd536552367d8a, user_id=4c1fd6d99758441486315678845b4673, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.381754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066320bd-b8be-499d-8934-fab39b0731ae admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.403333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1159/2283] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:40 2022] PUT /identity/v3/projects/bdf51e13c4b9456e95bd536552367d8a/users/4c1fd6d99758441486315678845b4673/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:40.415513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cae1a-931b-4332-90fb-50ede6203fb1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.415927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cae1a-931b-4332-90fb-50ede6203fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.416265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cae1a-931b-4332-90fb-50ede6203fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.451781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-414cae1a-931b-4332-90fb-50ede6203fb1 None None] MFA Rules not processed for user `4c1fd6d99758441486315678845b4673`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:40.453255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-414cae1a-931b-4332-90fb-50ede6203fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:40.471597 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a1367-7134-472f-8bbe-ba1b0bd32471 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.525676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1160/2284] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:40.534165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3c5cb-e4d8-4596-9dd2-40ae6ae1ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.534842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3c5cb-e4d8-4596-9dd2-40ae6ae1ebbf admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.535121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3c5cb-e4d8-4596-9dd2-40ae6ae1ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.535388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3c5cb-e4d8-4596-9dd2-40ae6ae1ebbf admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.536095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3c5cb-e4d8-4596-9dd2-40ae6ae1ebbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.537984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3c5cb-e4d8-4596-9dd2-40ae6ae1ebbf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.538915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cae1a-931b-4332-90fb-50ede6203fb1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.540512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-414cae1a-931b-4332-90fb-50ede6203fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:40.566739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1125/2285] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:40 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:40.571108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3c5cb-e4d8-4596-9dd2-40ae6ae1ebbf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.581556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3c5cb-e4d8-4596-9dd2-40ae6ae1ebbf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.586546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b349225-77c7-43bb-8b74-c8442ef06ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.587291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b349225-77c7-43bb-8b74-c8442ef06ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.587629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b349225-77c7-43bb-8b74-c8442ef06ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.587980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b349225-77c7-43bb-8b74-c8442ef06ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.588695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b349225-77c7-43bb-8b74-c8442ef06ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.590490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b349225-77c7-43bb-8b74-c8442ef06ee5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.605374 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1126/2286] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:40.606427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b349225-77c7-43bb-8b74-c8442ef06ee5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.616214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cae1a-931b-4332-90fb-50ede6203fb1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.616429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b349225-77c7-43bb-8b74-c8442ef06ee5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.620454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66585f-372e-4bd7-ba38-4abe6b1223ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.621055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66585f-372e-4bd7-ba38-4abe6b1223ab service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.621331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66585f-372e-4bd7-ba38-4abe6b1223ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.621600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66585f-372e-4bd7-ba38-4abe6b1223ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.622003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e413296-ebdf-466a-8de2-c170ae7044aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.622797 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413296-ebdf-466a-8de2-c170ae7044aa admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.623004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413296-ebdf-466a-8de2-c170ae7044aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.623226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413296-ebdf-466a-8de2-c170ae7044aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.623949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e413296-ebdf-466a-8de2-c170ae7044aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.625810 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e413296-ebdf-466a-8de2-c170ae7044aa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.632458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66585f-372e-4bd7-ba38-4abe6b1223ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.633900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66585f-372e-4bd7-ba38-4abe6b1223ab service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.641279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1127/2287] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:40.642380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e413296-ebdf-466a-8de2-c170ae7044aa admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.643794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1161/2288] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:40.654373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e413296-ebdf-466a-8de2-c170ae7044aa admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.659413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec72629-c6c4-45be-b647-b29ccfca90a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.660181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec72629-c6c4-45be-b647-b29ccfca90a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.660461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec72629-c6c4-45be-b647-b29ccfca90a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.660732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec72629-c6c4-45be-b647-b29ccfca90a1 admin admin] PATH_INFO: `/v3/projects/343e80b068df492ca89a5812620c4125/users/d4ba548ad2c44c598cd11c7e8fea2f1c/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.676832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec72629-c6c4-45be-b647-b29ccfca90a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=343e80b068df492ca89a5812620c4125, user_id=d4ba548ad2c44c598cd11c7e8fea2f1c, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.679793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec72629-c6c4-45be-b647-b29ccfca90a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.695289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1128/2289] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:40 2022] PUT /identity/v3/projects/343e80b068df492ca89a5812620c4125/users/d4ba548ad2c44c598cd11c7e8fea2f1c/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:40.707632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f8414-adeb-406a-acab-85181a49acfd None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.708948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f8414-adeb-406a-acab-85181a49acfd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.710034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f8414-adeb-406a-acab-85181a49acfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.759188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a64f8414-adeb-406a-acab-85181a49acfd None None] MFA Rules not processed for user `d4ba548ad2c44c598cd11c7e8fea2f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:40.762798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a64f8414-adeb-406a-acab-85181a49acfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:40.815627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:40.850905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1162/2290] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:36:40.855908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec72629-c6c4-45be-b647-b29ccfca90a1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.922501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:40.941672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec72629-c6c4-45be-b647-b29ccfca90a1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:40.947116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e654-9ff6-48c8-8a77-51695cd2c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:40.947677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e654-9ff6-48c8-8a77-51695cd2c9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:40.947996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e654-9ff6-48c8-8a77-51695cd2c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:40.948212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e654-9ff6-48c8-8a77-51695cd2c9d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:40.948850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07e654-9ff6-48c8-8a77-51695cd2c9d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:40.950815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07e654-9ff6-48c8-8a77-51695cd2c9d0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:40.981194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1129/2291] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:40 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:40.985797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f8414-adeb-406a-acab-85181a49acfd None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:40.997508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f8414-adeb-406a-acab-85181a49acfd None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.060178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704735b-ae7d-48d3-93f1-fc2b034e14c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.060888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704735b-ae7d-48d3-93f1-fc2b034e14c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.061191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704735b-ae7d-48d3-93f1-fc2b034e14c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.061463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704735b-ae7d-48d3-93f1-fc2b034e14c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.062456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704735b-ae7d-48d3-93f1-fc2b034e14c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.065200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704735b-ae7d-48d3-93f1-fc2b034e14c6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.103647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1163/2292] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:40 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:41.106404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e654-9ff6-48c8-8a77-51695cd2c9d0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.117741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e654-9ff6-48c8-8a77-51695cd2c9d0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.122841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9b352-fde7-45f2-a59e-9c5efcc3a8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.123450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9b352-fde7-45f2-a59e-9c5efcc3a8da admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.123759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9b352-fde7-45f2-a59e-9c5efcc3a8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.124018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9b352-fde7-45f2-a59e-9c5efcc3a8da admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.124733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9b352-fde7-45f2-a59e-9c5efcc3a8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.126544 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9b352-fde7-45f2-a59e-9c5efcc3a8da admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.139029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1130/2293] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:41.143563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704735b-ae7d-48d3-93f1-fc2b034e14c6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.154237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704735b-ae7d-48d3-93f1-fc2b034e14c6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.161645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30594d7-0d73-4eaf-af73-d2c6524ba1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.162572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30594d7-0d73-4eaf-af73-d2c6524ba1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.162892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30594d7-0d73-4eaf-af73-d2c6524ba1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.163281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30594d7-0d73-4eaf-af73-d2c6524ba1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.164091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30594d7-0d73-4eaf-af73-d2c6524ba1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.165819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30594d7-0d73-4eaf-af73-d2c6524ba1cd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.178889 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1164/2294] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:41.181455 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9b352-fde7-45f2-a59e-9c5efcc3a8da admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.193606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9b352-fde7-45f2-a59e-9c5efcc3a8da admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.199942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd33c3-2a7f-4ac7-b23e-6326bdf1f14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.199942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd33c3-2a7f-4ac7-b23e-6326bdf1f14b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.199942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd33c3-2a7f-4ac7-b23e-6326bdf1f14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.199942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd33c3-2a7f-4ac7-b23e-6326bdf1f14b admin admin] PATH_INFO: `/v3/projects/f0e637a5d1d94733a8e59f70545a44bb/users/20ad4102a5624f6bb6ab1e1427aff123/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.218109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd33c3-2a7f-4ac7-b23e-6326bdf1f14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e637a5d1d94733a8e59f70545a44bb, user_id=20ad4102a5624f6bb6ab1e1427aff123, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.221306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd33c3-2a7f-4ac7-b23e-6326bdf1f14b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.239360 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1131/2295] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:41 2022] PUT /identity/v3/projects/f0e637a5d1d94733a8e59f70545a44bb/users/20ad4102a5624f6bb6ab1e1427aff123/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:41.252016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a053-4888-4eb2-9421-6998da0144c0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.252448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a053-4888-4eb2-9421-6998da0144c0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.252727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6a053-4888-4eb2-9421-6998da0144c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.291488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-daf6a053-4888-4eb2-9421-6998da0144c0 None None] MFA Rules not processed for user `20ad4102a5624f6bb6ab1e1427aff123`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:41.292814 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-daf6a053-4888-4eb2-9421-6998da0144c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:41.368410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1165/2296] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:41.375056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd33c3-2a7f-4ac7-b23e-6326bdf1f14b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.445421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd33c3-2a7f-4ac7-b23e-6326bdf1f14b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.450751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4ee0e-e404-4712-a958-aca614c4cde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.451358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4ee0e-e404-4712-a958-aca614c4cde9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.451636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4ee0e-e404-4712-a958-aca614c4cde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.451897 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4ee0e-e404-4712-a958-aca614c4cde9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.452524 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4ee0e-e404-4712-a958-aca614c4cde9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.454639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4ee0e-e404-4712-a958-aca614c4cde9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.483638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1132/2297] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:41 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:41.487832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6a053-4888-4eb2-9421-6998da0144c0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.498952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6a053-4888-4eb2-9421-6998da0144c0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.544628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:41.558269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b3aa5-ee99-4375-adfe-a38261c79408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.558923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b3aa5-ee99-4375-adfe-a38261c79408 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.559202 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b3aa5-ee99-4375-adfe-a38261c79408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.559463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b3aa5-ee99-4375-adfe-a38261c79408 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.560089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b3aa5-ee99-4375-adfe-a38261c79408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.562101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b3aa5-ee99-4375-adfe-a38261c79408 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.585326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1166/2298] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:41 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:41.589348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4ee0e-e404-4712-a958-aca614c4cde9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.600784 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4ee0e-e404-4712-a958-aca614c4cde9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.606232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82187f-c92b-4e9c-9308-e17ad878c8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.606880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82187f-c92b-4e9c-9308-e17ad878c8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.607160 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82187f-c92b-4e9c-9308-e17ad878c8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.607421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82187f-c92b-4e9c-9308-e17ad878c8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.608067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82187f-c92b-4e9c-9308-e17ad878c8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.609732 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82187f-c92b-4e9c-9308-e17ad878c8f5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.621506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1133/2299] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:41.625894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b3aa5-ee99-4375-adfe-a38261c79408 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.637071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b3aa5-ee99-4375-adfe-a38261c79408 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.642038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a37f6b-b207-4f4f-a8ef-ef96a4189297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.642697 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a37f6b-b207-4f4f-a8ef-ef96a4189297 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.643024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a37f6b-b207-4f4f-a8ef-ef96a4189297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.643299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a37f6b-b207-4f4f-a8ef-ef96a4189297 admin admin] PATH_INFO: `/v3/projects/1b5cc729d1334d37a5c4b4f30ae75a95/users/f1b16672ab5149f89ab0c2cc156b963c/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.668263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a37f6b-b207-4f4f-a8ef-ef96a4189297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b5cc729d1334d37a5c4b4f30ae75a95, user_id=f1b16672ab5149f89ab0c2cc156b963c, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.671027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a37f6b-b207-4f4f-a8ef-ef96a4189297 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.686648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1167/2300] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:41 2022] PUT /identity/v3/projects/1b5cc729d1334d37a5c4b4f30ae75a95/users/f1b16672ab5149f89ab0c2cc156b963c/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:41.690804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82187f-c92b-4e9c-9308-e17ad878c8f5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.700725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82187f-c92b-4e9c-9308-e17ad878c8f5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.763827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1b40-6969-4b40-aa69-89110906695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.764449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1b40-6969-4b40-aa69-89110906695a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.764724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1b40-6969-4b40-aa69-89110906695a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.764993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1b40-6969-4b40-aa69-89110906695a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.765652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f1b40-6969-4b40-aa69-89110906695a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.767543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f1b40-6969-4b40-aa69-89110906695a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.778778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1134/2301] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:41.783283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a37f6b-b207-4f4f-a8ef-ef96a4189297 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.795600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a37f6b-b207-4f4f-a8ef-ef96a4189297 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:41.800261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:41.801001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.801340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.801873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] PATH_INFO: `/v3/projects/1b5cc729d1334d37a5c4b4f30ae75a95/users/f1b16672ab5149f89ab0c2cc156b963c/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.804373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b5cc729d1334d37a5c4b4f30ae75a95, user_id=f1b16672ab5149f89ab0c2cc156b963c, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:41.808618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:41.824403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1168/2302] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:41 2022] PUT /identity/v3/projects/1b5cc729d1334d37a5c4b4f30ae75a95/users/f1b16672ab5149f89ab0c2cc156b963c/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:41.837785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cc9b1-edbe-400d-a9ff-dc1240055651 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:41.838397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cc9b1-edbe-400d-a9ff-dc1240055651 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:41.838980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cc9b1-edbe-400d-a9ff-dc1240055651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:41.861648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:41.872432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-db1cc9b1-edbe-400d-a9ff-dc1240055651 None None] MFA Rules not processed for user `f1b16672ab5149f89ab0c2cc156b963c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:41.874421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-db1cc9b1-edbe-400d-a9ff-dc1240055651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:41.951620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1135/2303] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:41.963902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:41.965626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:42.026645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedd41a-5fc7-4bf8-b216-6c8f70272c33 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:42.030920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179581b2-3c8c-482c-9d15-afd001695326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:42.031537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179581b2-3c8c-482c-9d15-afd001695326 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:42.031873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179581b2-3c8c-482c-9d15-afd001695326 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:42.032140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179581b2-3c8c-482c-9d15-afd001695326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:42.093227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179581b2-3c8c-482c-9d15-afd001695326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:42.094699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179581b2-3c8c-482c-9d15-afd001695326 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:42.103805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1169/2304] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:51.948522 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=30 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:52.101639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:52.134945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cc9b1-edbe-400d-a9ff-dc1240055651 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:52.144499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cc9b1-edbe-400d-a9ff-dc1240055651 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:52.149458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2870908-0349-4ad6-8e02-15826454cb2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:52.150748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2870908-0349-4ad6-8e02-15826454cb2d service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:52.151120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2870908-0349-4ad6-8e02-15826454cb2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:52.151443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2870908-0349-4ad6-8e02-15826454cb2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:52.229648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2870908-0349-4ad6-8e02-15826454cb2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:52.231231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2870908-0349-4ad6-8e02-15826454cb2d service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:52.241491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1136/2305] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:52.894028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179581b2-3c8c-482c-9d15-afd001695326 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:52.903438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179581b2-3c8c-482c-9d15-afd001695326 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:52.908037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435587de-02c0-4e5d-8989-51db4b077fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:52.908729 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435587de-02c0-4e5d-8989-51db4b077fba service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:52.909068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435587de-02c0-4e5d-8989-51db4b077fba service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:52.909313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435587de-02c0-4e5d-8989-51db4b077fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:52.921355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435587de-02c0-4e5d-8989-51db4b077fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:52.922875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435587de-02c0-4e5d-8989-51db4b077fba service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:52.934674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1170/2306] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:53.430069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2870908-0349-4ad6-8e02-15826454cb2d service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:53.521391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2870908-0349-4ad6-8e02-15826454cb2d service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:53.532512 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:53.533398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:53.534151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:53.534612 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] PATH_INFO: `/v3/users/4c1fd6d99758441486315678845b4673` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:53.537843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c1fd6d99758441486315678845b4673)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:53.541182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:53.628636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c1fd6d99758441486315678845b4673'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:53.643174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c1fd6d99758441486315678845b4673'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:53.644592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c1fd6d99758441486315678845b4673'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:53.663878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c1fd6d99758441486315678845b4673'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:53.664821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c1fd6d99758441486315678845b4673'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:53.679979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1137/2307] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:53 2022] DELETE /identity/v3/users/4c1fd6d99758441486315678845b4673 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:53.685008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1549-d672-472a-8d77-3649b5bcbcf0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:53.685511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1549-d672-472a-8d77-3649b5bcbcf0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:53.685804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1549-d672-472a-8d77-3649b5bcbcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:53.707976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f7fa1549-d672-472a-8d77-3649b5bcbcf0 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:53.709303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f7fa1549-d672-472a-8d77-3649b5bcbcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:53.779690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1171/2308] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:53.790073 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:53.791609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:53.812949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:53.856263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b5515-efe4-4064-ba75-5a5522229298 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:53.859908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8746e-d3a8-4b9f-a80f-00489fbd119c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:53.860458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8746e-d3a8-4b9f-a80f-00489fbd119c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:53.860734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8746e-d3a8-4b9f-a80f-00489fbd119c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:53.861001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8746e-d3a8-4b9f-a80f-00489fbd119c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:53.870338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8746e-d3a8-4b9f-a80f-00489fbd119c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:53.871785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8746e-d3a8-4b9f-a80f-00489fbd119c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:53.881757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1138/2309] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:54.169043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa1549-d672-472a-8d77-3649b5bcbcf0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:54.171233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f7fa1549-d672-472a-8d77-3649b5bcbcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:54.184110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa1549-d672-472a-8d77-3649b5bcbcf0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:54.189391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:54.190105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:54.190480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:54.191006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] PATH_INFO: `/v3/projects/bdf51e13c4b9456e95bd536552367d8a` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:54.192170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf51e13c4b9456e95bd536552367d8a)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:54.195811 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:54.250608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf51e13c4b9456e95bd536552367d8a', 'request_id': 'req-e6aab3f1-094f-4202-b346-dbb756a337dc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:54.259416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf51e13c4b9456e95bd536552367d8a', 'request_id': 'req-e6aab3f1-094f-4202-b346-dbb756a337dc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:54.266777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf51e13c4b9456e95bd536552367d8a', 'request_id': 'req-e6aab3f1-094f-4202-b346-dbb756a337dc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:54.299500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] The token cache is being invalidate because project bdf51e13c4b9456e95bd536552367d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:54.299913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:54.300866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6aab3f1-094f-4202-b346-dbb756a337dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:54.303165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1172/2310] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:54 2022] DELETE /identity/v3/projects/bdf51e13c4b9456e95bd536552367d8a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:54.309638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c736-b191-49f9-9285-5632ead47818 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:54.309638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c736-b191-49f9-9285-5632ead47818 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:54.309638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c736-b191-49f9-9285-5632ead47818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:54.330291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c364c736-b191-49f9-9285-5632ead47818 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:54.331564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c364c736-b191-49f9-9285-5632ead47818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:54.529001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1139/2311] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:54.542915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed534b3-5eb8-4501-b60e-1acc68f30407 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:54.545162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed534b3-5eb8-4501-b60e-1acc68f30407 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:54.545586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed534b3-5eb8-4501-b60e-1acc68f30407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:54.590400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5ed534b3-5eb8-4501-b60e-1acc68f30407 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:54.591685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5ed534b3-5eb8-4501-b60e-1acc68f30407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:54.617121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1173/2312] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:54.617762 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364c736-b191-49f9-9285-5632ead47818 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:54.629589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364c736-b191-49f9-9285-5632ead47818 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:54.644372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:54.645131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:54.645403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:54.645671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:54.646326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:54.646910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:54.648712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:54.678376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1140/2313] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:54.682898 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed534b3-5eb8-4501-b60e-1acc68f30407 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:54.701417 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed534b3-5eb8-4501-b60e-1acc68f30407 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:54.709788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afe2e5-de94-486f-950f-e27bcef8515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:54.712700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afe2e5-de94-486f-950f-e27bcef8515d admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:54.712993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afe2e5-de94-486f-950f-e27bcef8515d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:54.713265 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afe2e5-de94-486f-950f-e27bcef8515d admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:54.714120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afe2e5-de94-486f-950f-e27bcef8515d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:54.719869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afe2e5-de94-486f-950f-e27bcef8515d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:54.799627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1174/2314] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:54.811018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:54.815923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a159e-1fca-4299-8f04-e682e7435ee7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:54.878691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826269f-c7ed-4757-a4e0-98edaff71aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:54.879398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826269f-c7ed-4757-a4e0-98edaff71aeb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:54.879677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826269f-c7ed-4757-a4e0-98edaff71aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:54.879944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826269f-c7ed-4757-a4e0-98edaff71aeb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:54.880745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826269f-c7ed-4757-a4e0-98edaff71aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:54.883083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826269f-c7ed-4757-a4e0-98edaff71aeb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:54.911021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1141/2315] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:54 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:36:54.915212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afe2e5-de94-486f-950f-e27bcef8515d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:54.929736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afe2e5-de94-486f-950f-e27bcef8515d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:54.934750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9548194-cbf7-4d97-ae54-b9ca51705b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:54.935463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9548194-cbf7-4d97-ae54-b9ca51705b15 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:54.935743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9548194-cbf7-4d97-ae54-b9ca51705b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:54.936018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9548194-cbf7-4d97-ae54-b9ca51705b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:54.936669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9548194-cbf7-4d97-ae54-b9ca51705b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:54.938305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9548194-cbf7-4d97-ae54-b9ca51705b15 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:54.952874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1175/2316] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:54.959919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826269f-c7ed-4757-a4e0-98edaff71aeb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:54.970210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826269f-c7ed-4757-a4e0-98edaff71aeb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:54.976155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741bdeb-17ec-4f32-8550-112f4b8461c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:54.976830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741bdeb-17ec-4f32-8550-112f4b8461c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:54.977105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741bdeb-17ec-4f32-8550-112f4b8461c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:54.977370 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741bdeb-17ec-4f32-8550-112f4b8461c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:54.978049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741bdeb-17ec-4f32-8550-112f4b8461c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:54.979681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741bdeb-17ec-4f32-8550-112f4b8461c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:54.991257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1142/2317] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:54.999096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9548194-cbf7-4d97-ae54-b9ca51705b15 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:55.009339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9548194-cbf7-4d97-ae54-b9ca51705b15 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:55.014635 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:55.015324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:55.015602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:55.015877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] PATH_INFO: `/v3/projects/52fbe77f79a5449b8fb2bfef82360b94/users/503ba8b1d3f64186b188bed35c4baefe/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:55.032423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=52fbe77f79a5449b8fb2bfef82360b94, user_id=503ba8b1d3f64186b188bed35c4baefe, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:55.035158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:55.051252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1176/2318] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:54 2022] PUT /identity/v3/projects/52fbe77f79a5449b8fb2bfef82360b94/users/503ba8b1d3f64186b188bed35c4baefe/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:55.063359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647121e-9989-43db-8830-fa263a3d8606 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:55.063359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647121e-9989-43db-8830-fa263a3d8606 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:55.063359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647121e-9989-43db-8830-fa263a3d8606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:55.083573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a647121e-9989-43db-8830-fa263a3d8606 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:55.084785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a647121e-9989-43db-8830-fa263a3d8606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:55.106660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:55.185410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1143/2319] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:55.215314 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:55.217294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:55.232295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:55.316240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed1673-2fda-48b1-9786-4751eb2bd02d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:55.321688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:55.322824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:55.323360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:55.323709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:55.336474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:55.337926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:55.347498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1177/2320] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:56.300533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45448a6-dbc2-4503-b019-9058700f0222 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:56.301468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45448a6-dbc2-4503-b019-9058700f0222 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:56.302505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45448a6-dbc2-4503-b019-9058700f0222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:56.338471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a45448a6-dbc2-4503-b019-9058700f0222 None None] MFA Rules not processed for user `503ba8b1d3f64186b188bed35c4baefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:56.341181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a45448a6-dbc2-4503-b019-9058700f0222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:56.428123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1144/2321] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:56.441214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:56.443316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:56.525101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc496fb3-25c9-4405-8c12-c55a1e991564 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:56.528664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea504-62b5-4947-acc5-1f353dcab336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:56.529266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea504-62b5-4947-acc5-1f353dcab336 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:56.529541 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea504-62b5-4947-acc5-1f353dcab336 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:56.529807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea504-62b5-4947-acc5-1f353dcab336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:56.537764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ea504-62b5-4947-acc5-1f353dcab336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:56.539221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ea504-62b5-4947-acc5-1f353dcab336 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:56.547724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1178/2322] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:57.129030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45448a6-dbc2-4503-b019-9058700f0222 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:57.131566 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a45448a6-dbc2-4503-b019-9058700f0222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:57.147726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45448a6-dbc2-4503-b019-9058700f0222 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:57.159805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:57.160593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:57.161553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:57.161893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] PATH_INFO: `/v3/users/d4ba548ad2c44c598cd11c7e8fea2f1c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:57.163349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ba548ad2c44c598cd11c7e8fea2f1c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:57.170199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:57.214024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ba548ad2c44c598cd11c7e8fea2f1c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.223614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ba548ad2c44c598cd11c7e8fea2f1c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.224334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ba548ad2c44c598cd11c7e8fea2f1c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.236957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ba548ad2c44c598cd11c7e8fea2f1c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.237649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ba548ad2c44c598cd11c7e8fea2f1c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.251536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1145/2323] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:57 2022] DELETE /identity/v3/users/d4ba548ad2c44c598cd11c7e8fea2f1c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:57.256514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb005728-246d-460e-aa13-d8a6f9e489fb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:57.256800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb005728-246d-460e-aa13-d8a6f9e489fb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:57.257094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb005728-246d-460e-aa13-d8a6f9e489fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:57.275915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fb005728-246d-460e-aa13-d8a6f9e489fb None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:57.277024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fb005728-246d-460e-aa13-d8a6f9e489fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:57.335756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1179/2324] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:57.345715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:57.347267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:57.380121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:57.406962 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db29cc8-0b1c-4552-b447-4c34e4c79605 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:57.411116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0528e1e-a1c1-44a5-b794-3c973738052c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:57.411478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0528e1e-a1c1-44a5-b794-3c973738052c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:57.411752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0528e1e-a1c1-44a5-b794-3c973738052c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:57.412020 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0528e1e-a1c1-44a5-b794-3c973738052c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:57.423820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0528e1e-a1c1-44a5-b794-3c973738052c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:57.426601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0528e1e-a1c1-44a5-b794-3c973738052c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:57.440855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1146/2325] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:36:57.777932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb005728-246d-460e-aa13-d8a6f9e489fb None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:57.779513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fb005728-246d-460e-aa13-d8a6f9e489fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:57.792326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb005728-246d-460e-aa13-d8a6f9e489fb None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:57.797194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:57.797837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:57.798113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:57.798600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] PATH_INFO: `/v3/projects/343e80b068df492ca89a5812620c4125` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:57.799914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=343e80b068df492ca89a5812620c4125)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:57.802263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:57.851134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:57.852722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '343e80b068df492ca89a5812620c4125', 'request_id': 'req-07618042-a9e7-4753-910b-b22ac279e0af'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.865272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '343e80b068df492ca89a5812620c4125', 'request_id': 'req-07618042-a9e7-4753-910b-b22ac279e0af'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.872152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '343e80b068df492ca89a5812620c4125', 'request_id': 'req-07618042-a9e7-4753-910b-b22ac279e0af'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.884372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] The token cache is being invalidate because project 343e80b068df492ca89a5812620c4125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:57.884731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.885445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:57.887176 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1180/2326] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:57 2022] DELETE /identity/v3/projects/343e80b068df492ca89a5812620c4125 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:57.891264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0528e1e-a1c1-44a5-b794-3c973738052c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:57.901285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0528e1e-a1c1-44a5-b794-3c973738052c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:57.958221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:57.958870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:57.959144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:57.959392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] PATH_INFO: `/v3/users/20ad4102a5624f6bb6ab1e1427aff123` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:57.961515 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ad4102a5624f6bb6ab1e1427aff123)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:57.963599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:58.016213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ad4102a5624f6bb6ab1e1427aff123'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.025106 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ad4102a5624f6bb6ab1e1427aff123'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.025790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ad4102a5624f6bb6ab1e1427aff123'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.039124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ad4102a5624f6bb6ab1e1427aff123'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.039902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ad4102a5624f6bb6ab1e1427aff123'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.052920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1147/2327] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:57 2022] DELETE /identity/v3/users/20ad4102a5624f6bb6ab1e1427aff123 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:58.364338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:58.365953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:58.379359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618042-a9e7-4753-910b-b22ac279e0af admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:58.438730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:58.439606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:58.439894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:58.440270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] PATH_INFO: `/v3/projects/f0e637a5d1d94733a8e59f70545a44bb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:58.441835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e637a5d1d94733a8e59f70545a44bb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:58.445166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:58.496875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e637a5d1d94733a8e59f70545a44bb', 'request_id': 'req-dfd626c8-a678-46da-93a8-1ec73e28f2d1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.506046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e637a5d1d94733a8e59f70545a44bb', 'request_id': 'req-dfd626c8-a678-46da-93a8-1ec73e28f2d1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.512548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e637a5d1d94733a8e59f70545a44bb', 'request_id': 'req-dfd626c8-a678-46da-93a8-1ec73e28f2d1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.525310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] The token cache is being invalidate because project f0e637a5d1d94733a8e59f70545a44bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:58.525614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.526261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.527800 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1181/2328] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:58 2022] DELETE /identity/v3/projects/f0e637a5d1d94733a8e59f70545a44bb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:36:58.531887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:58.533318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:58.547697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0848da-974d-42d2-8b05-5c11a18fc7cb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:58.596739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:58.608426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:58.609314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:58.609847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:58.610358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] PATH_INFO: `/v3/users/f1b16672ab5149f89ab0c2cc156b963c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:58.611569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b16672ab5149f89ab0c2cc156b963c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:58.613574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:58.661830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b16672ab5149f89ab0c2cc156b963c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.669269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b16672ab5149f89ab0c2cc156b963c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.669905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b16672ab5149f89ab0c2cc156b963c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.682340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b16672ab5149f89ab0c2cc156b963c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.682998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-90b0ef14-7cb3-418d-8182-9d73d1abb1dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b16672ab5149f89ab0c2cc156b963c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:58.694856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1148/2329] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:36:58 2022] DELETE /identity/v3/users/f1b16672ab5149f89ab0c2cc156b963c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:58.967552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:58.969217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:58.983962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd626c8-a678-46da-93a8-1ec73e28f2d1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:59.043256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:59.043892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.044171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.044443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] PATH_INFO: `/v3/projects/1b5cc729d1334d37a5c4b4f30ae75a95` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.045582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b5cc729d1334d37a5c4b4f30ae75a95)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:59.047546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:59.056410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:59.110765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b5cc729d1334d37a5c4b4f30ae75a95', 'request_id': 'req-2f76bad3-5131-438a-b103-29e2c514a1c1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:59.121430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b5cc729d1334d37a5c4b4f30ae75a95', 'request_id': 'req-2f76bad3-5131-438a-b103-29e2c514a1c1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:59.139772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b5cc729d1334d37a5c4b4f30ae75a95', 'request_id': 'req-2f76bad3-5131-438a-b103-29e2c514a1c1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:59.166882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] The token cache is being invalidate because project 1b5cc729d1334d37a5c4b4f30ae75a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:36:59.167491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:59.168229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2f76bad3-5131-438a-b103-29e2c514a1c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:36:59.170086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1182/2330] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:36:58 2022] DELETE /identity/v3/projects/1b5cc729d1334d37a5c4b4f30ae75a95 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:59.176254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82b3eb-e7ee-4c70-ad67-f520113c8a04 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.176540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82b3eb-e7ee-4c70-ad67-f520113c8a04 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.176916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82b3eb-e7ee-4c70-ad67-f520113c8a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.208246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4d82b3eb-e7ee-4c70-ad67-f520113c8a04 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:59.213352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d82b3eb-e7ee-4c70-ad67-f520113c8a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:59.340035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1149/2331] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:59.348824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f724a-12e2-4db3-a612-6f07a0b8b491 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.349602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f724a-12e2-4db3-a612-6f07a0b8b491 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.350220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f724a-12e2-4db3-a612-6f07a0b8b491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.375431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-739f724a-12e2-4db3-a612-6f07a0b8b491 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:59.376841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-739f724a-12e2-4db3-a612-6f07a0b8b491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:59.386351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1183/2332] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:59.391486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82b3eb-e7ee-4c70-ad67-f520113c8a04 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:59.412535 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82b3eb-e7ee-4c70-ad67-f520113c8a04 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:59.422763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:59.422763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.422763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.422763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.423219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:36:59.423219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:59.423219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:59.440906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1150/2333] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:36:59.446659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f724a-12e2-4db3-a612-6f07a0b8b491 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:59.475869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f724a-12e2-4db3-a612-6f07a0b8b491 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:59.492877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda4815-247b-4ced-a290-ea58299e5a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:59.493762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4815-247b-4ced-a290-ea58299e5a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.494195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4815-247b-4ced-a290-ea58299e5a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.494602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4815-247b-4ced-a290-ea58299e5a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.495888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda4815-247b-4ced-a290-ea58299e5a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:59.510124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda4815-247b-4ced-a290-ea58299e5a92 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:59.562119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1184/2334] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:36:59 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:59.563487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:59.574547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a68917-6725-4a8b-b490-f8ab53d8d90c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:59.633929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:36:59.640849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fdb98-bc60-40f1-bfc5-8fefaafb68a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:59.641733 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fdb98-bc60-40f1-bfc5-8fefaafb68a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.642284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fdb98-bc60-40f1-bfc5-8fefaafb68a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.642684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fdb98-bc60-40f1-bfc5-8fefaafb68a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.643604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fdb98-bc60-40f1-bfc5-8fefaafb68a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:59.646241 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fdb98-bc60-40f1-bfc5-8fefaafb68a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:59.673000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1151/2335] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:36:59 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:36:59.675116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda4815-247b-4ced-a290-ea58299e5a92 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:59.685666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda4815-247b-4ced-a290-ea58299e5a92 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:59.690492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5ff5-942e-4af0-9031-a8d657a58718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:59.691144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5ff5-942e-4af0-9031-a8d657a58718 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.691420 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5ff5-942e-4af0-9031-a8d657a58718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.691688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5ff5-942e-4af0-9031-a8d657a58718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.692357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f5ff5-942e-4af0-9031-a8d657a58718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:59.694115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f5ff5-942e-4af0-9031-a8d657a58718 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:59.717997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1185/2336] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:36:59.722233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fdb98-bc60-40f1-bfc5-8fefaafb68a1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:59.734196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fdb98-bc60-40f1-bfc5-8fefaafb68a1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:59.740888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb625deb-4770-456d-9432-f30caebd3428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:59.741502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb625deb-4770-456d-9432-f30caebd3428 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.741780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb625deb-4770-456d-9432-f30caebd3428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.742045 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb625deb-4770-456d-9432-f30caebd3428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.742929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb625deb-4770-456d-9432-f30caebd3428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:59.745353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb625deb-4770-456d-9432-f30caebd3428 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:59.762534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1152/2337] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:36:59.764244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5ff5-942e-4af0-9031-a8d657a58718 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:59.775344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5ff5-942e-4af0-9031-a8d657a58718 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:36:59.780405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:36:59.781071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.781338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.781607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] PATH_INFO: `/v3/projects/c707ecbc947f49debcd248a815c69f41/users/e5541503f3834866aa1d4801fefda98d/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.797032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c707ecbc947f49debcd248a815c69f41, user_id=e5541503f3834866aa1d4801fefda98d, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:36:59.799731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:36:59.813903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1186/2338] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:36:59 2022] PUT /identity/v3/projects/c707ecbc947f49debcd248a815c69f41/users/e5541503f3834866aa1d4801fefda98d/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:36:59.824577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718597d-2404-4c8b-975f-99d7a852ddbd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:36:59.824866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718597d-2404-4c8b-975f-99d7a852ddbd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:36:59.825141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718597d-2404-4c8b-975f-99d7a852ddbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:36:59.855285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4718597d-2404-4c8b-975f-99d7a852ddbd None None] MFA Rules not processed for user `e5541503f3834866aa1d4801fefda98d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:36:59.856514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4718597d-2404-4c8b-975f-99d7a852ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:36:59.931904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1153/2339] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:36:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:36:59.944503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:36:59.946644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:00.023804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb47daa-98b1-4cb6-b1eb-9d3205186586 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:00.024075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb47daa-98b1-4cb6-b1eb-9d3205186586 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:00.024352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb47daa-98b1-4cb6-b1eb-9d3205186586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:00.035370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e20378-3209-490f-a2bd-ef7010f2df9f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:00.040354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:00.041242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:00.041876 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:00.042323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:00.046400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ddb47daa-98b1-4cb6-b1eb-9d3205186586 None None] MFA Rules not processed for user `503ba8b1d3f64186b188bed35c4baefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:00.047664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ddb47daa-98b1-4cb6-b1eb-9d3205186586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:00.053375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:00.055885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:00.086050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1187/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:36:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:00.131991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1154/2341] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:05.663351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:05.665027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:05.765750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:05.832073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc123-d15a-4016-a721-ac611cf7a64e service glance] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:05.840931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:05.841911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:05.842699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:05.843177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] PATH_INFO: `/v3/users/dbd6e4fc6e0f41d6942a6a1bd885cd59` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:05.844816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd6e4fc6e0f41d6942a6a1bd885cd59)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:05.847214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:05.883739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ab0e-b255-4c3f-827c-1681e87f5668 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:05.884320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ab0e-b255-4c3f-827c-1681e87f5668 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:05.884508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ab0e-b255-4c3f-827c-1681e87f5668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:05.932753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d7f2ab0e-b255-4c3f-827c-1681e87f5668 None None] MFA Rules not processed for user `863e457bd6fd4bff82f552e63ca1034c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:05.935376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d7f2ab0e-b255-4c3f-827c-1681e87f5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:05.964492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd6e4fc6e0f41d6942a6a1bd885cd59'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:05.987405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd6e4fc6e0f41d6942a6a1bd885cd59'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:05.988827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd6e4fc6e0f41d6942a6a1bd885cd59'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:06.005667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd6e4fc6e0f41d6942a6a1bd885cd59'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:06.006546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-329bb5e4-65cd-4b68-9e1b-74efb0a4500f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd6e4fc6e0f41d6942a6a1bd885cd59'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:06.023548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1188/2342] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:37:05 2022] DELETE /identity/v3/users/dbd6e4fc6e0f41d6942a6a1bd885cd59 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:06.033672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1155/2343] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:06.052443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18aad0-ff19-41d8-ad8d-31c376985a16 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:06.052725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18aad0-ff19-41d8-ad8d-31c376985a16 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:06.053001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18aad0-ff19-41d8-ad8d-31c376985a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:06.075201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8b18aad0-ff19-41d8-ad8d-31c376985a16 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:06.076489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8b18aad0-ff19-41d8-ad8d-31c376985a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:06.143405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1189/2344] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:06.154803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2ab0e-b255-4c3f-827c-1681e87f5668 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:06.156320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d7f2ab0e-b255-4c3f-827c-1681e87f5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:06.183013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:06.231466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2ab0e-b255-4c3f-827c-1681e87f5668 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:06.356767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59466905-a4ec-4768-b9d3-54283f3077ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:06.359695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466905-a4ec-4768-b9d3-54283f3077ad service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:06.359695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466905-a4ec-4768-b9d3-54283f3077ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:06.359695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466905-a4ec-4768-b9d3-54283f3077ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:06.373056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59466905-a4ec-4768-b9d3-54283f3077ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:06.376377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59466905-a4ec-4768-b9d3-54283f3077ad service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:06.389307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1156/2345] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:07.111805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18aad0-ff19-41d8-ad8d-31c376985a16 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:07.113581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8b18aad0-ff19-41d8-ad8d-31c376985a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:07.126251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18aad0-ff19-41d8-ad8d-31c376985a16 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:07.132491 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:07.133584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.134652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.135449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] PATH_INFO: `/v3/projects/5bdf070482b344edbc329657aa0b8a20` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.146863 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bdf070482b344edbc329657aa0b8a20)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:07.150698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:07.241593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bdf070482b344edbc329657aa0b8a20', 'request_id': 'req-e20369ae-a02c-47d8-a89f-7bb39cf43c52'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:07.244055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:07.263822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bdf070482b344edbc329657aa0b8a20', 'request_id': 'req-e20369ae-a02c-47d8-a89f-7bb39cf43c52'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:07.274384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bdf070482b344edbc329657aa0b8a20', 'request_id': 'req-e20369ae-a02c-47d8-a89f-7bb39cf43c52'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:07.290442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] The token cache is being invalidate because project 5bdf070482b344edbc329657aa0b8a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:37:07.291057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:07.292151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e20369ae-a02c-47d8-a89f-7bb39cf43c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:07.295155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1190/2346] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:37:07 2022] DELETE /identity/v3/projects/5bdf070482b344edbc329657aa0b8a20 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:07.303108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763a13-2e5d-4796-97de-ed4fff9b673f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.304308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763a13-2e5d-4796-97de-ed4fff9b673f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.304816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763a13-2e5d-4796-97de-ed4fff9b673f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.335744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4763a13-2e5d-4796-97de-ed4fff9b673f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:07.336996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4763a13-2e5d-4796-97de-ed4fff9b673f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:07.426214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1157/2347] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:07.428794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03cfcc-73c0-4dcc-8a26-0e98b9f90633 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.429222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03cfcc-73c0-4dcc-8a26-0e98b9f90633 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.429894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03cfcc-73c0-4dcc-8a26-0e98b9f90633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.463397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7a03cfcc-73c0-4dcc-8a26-0e98b9f90633 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:07.464826 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7a03cfcc-73c0-4dcc-8a26-0e98b9f90633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:07.475418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1191/2348] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:07.480080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4763a13-2e5d-4796-97de-ed4fff9b673f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:07.492178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4763a13-2e5d-4796-97de-ed4fff9b673f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:07.497768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:07.498432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.498809 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.500361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.501053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:37:07.501437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:07.503412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:07.516118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1158/2349] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:37:07.522447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03cfcc-73c0-4dcc-8a26-0e98b9f90633 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:07.534119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03cfcc-73c0-4dcc-8a26-0e98b9f90633 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:07.540409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d11f-28b6-4af0-8951-c16eb188a86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:07.541216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d11f-28b6-4af0-8951-c16eb188a86f admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.541488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d11f-28b6-4af0-8951-c16eb188a86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.541755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d11f-28b6-4af0-8951-c16eb188a86f admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.542379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68d11f-28b6-4af0-8951-c16eb188a86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:07.544357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68d11f-28b6-4af0-8951-c16eb188a86f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:07.586894 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1192/2350] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:37:07 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:37:07.590708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:07.605300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967397f0-f0ba-421d-a0d1-b3bbfc73cfc8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:07.678932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e5bfa-181d-4817-8206-229846a2489a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:07.679933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e5bfa-181d-4817-8206-229846a2489a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.680224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e5bfa-181d-4817-8206-229846a2489a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.680496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e5bfa-181d-4817-8206-229846a2489a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.681182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e5bfa-181d-4817-8206-229846a2489a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:07.683173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e5bfa-181d-4817-8206-229846a2489a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:07.694792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:07.711581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1159/2351] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:37:07 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:07.715516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d11f-28b6-4af0-8951-c16eb188a86f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:07.728886 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d11f-28b6-4af0-8951-c16eb188a86f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:07.735452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268bf07-4e6a-41a4-851e-3ba97bf2d048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:07.736560 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268bf07-4e6a-41a4-851e-3ba97bf2d048 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.736843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268bf07-4e6a-41a4-851e-3ba97bf2d048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.738370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268bf07-4e6a-41a4-851e-3ba97bf2d048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.739478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268bf07-4e6a-41a4-851e-3ba97bf2d048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:07.741518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268bf07-4e6a-41a4-851e-3ba97bf2d048 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:07.767582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1193/2352] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:07.771526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e5bfa-181d-4817-8206-229846a2489a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:07.796744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e5bfa-181d-4817-8206-229846a2489a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:07.802442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17661c-f51a-46dd-bcf9-8b0969a045e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:07.803194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17661c-f51a-46dd-bcf9-8b0969a045e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.803563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17661c-f51a-46dd-bcf9-8b0969a045e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.803905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17661c-f51a-46dd-bcf9-8b0969a045e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.804637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17661c-f51a-46dd-bcf9-8b0969a045e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:07.806334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17661c-f51a-46dd-bcf9-8b0969a045e5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:07.819260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1160/2353] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:37:07.823467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268bf07-4e6a-41a4-851e-3ba97bf2d048 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:07.833722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268bf07-4e6a-41a4-851e-3ba97bf2d048 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:07.839124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:07.839778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.840076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.840345 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] PATH_INFO: `/v3/projects/b35a44f3c0ba4230bac2bcb9d4e8ec08/users/fffc906c79ba418cbc52e5db5ca991ca/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.857291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35a44f3c0ba4230bac2bcb9d4e8ec08, user_id=fffc906c79ba418cbc52e5db5ca991ca, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:07.860289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:07.875424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1194/2354] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:37:07 2022] PUT /identity/v3/projects/b35a44f3c0ba4230bac2bcb9d4e8ec08/users/fffc906c79ba418cbc52e5db5ca991ca/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:37:07.880569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf4c9c-09d0-485e-915d-8cb5c67e5875 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:07.881109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf4c9c-09d0-485e-915d-8cb5c67e5875 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:07.881384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf4c9c-09d0-485e-915d-8cb5c67e5875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:07.901308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5ecf4c9c-09d0-485e-915d-8cb5c67e5875 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:07.902521 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5ecf4c9c-09d0-485e-915d-8cb5c67e5875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:07.976859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1161/2355] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:07.987765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:07.992692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:08.116409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac437ad9-9d26-4c43-aae5-df2d142f937d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:08.120349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:08.120986 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:08.121262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:08.121531 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:08.137603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:08.139644 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:08.150641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1195/2356] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:09.625201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b29ae1-80f6-4812-a8fc-79cbb279d879 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:09.626153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b29ae1-80f6-4812-a8fc-79cbb279d879 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:09.626750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b29ae1-80f6-4812-a8fc-79cbb279d879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:09.655994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-84b29ae1-80f6-4812-a8fc-79cbb279d879 None None] MFA Rules not processed for user `fffc906c79ba418cbc52e5db5ca991ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:09.657270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-84b29ae1-80f6-4812-a8fc-79cbb279d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:09.805537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1162/2357] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:09.813336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:09.814835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:09.870372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b29ae1-80f6-4812-a8fc-79cbb279d879 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:09.872053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-84b29ae1-80f6-4812-a8fc-79cbb279d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:09.898384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b29ae1-80f6-4812-a8fc-79cbb279d879 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:09.910772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:09.911650 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:09.911909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:09.912328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] PATH_INFO: `/v3/users/6b46b894e2ec49e882572cd50ff6bd47` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:09.914305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b46b894e2ec49e882572cd50ff6bd47)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:09.918682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:09.989418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39abe18-fdb7-40c0-ab89-d46685134667 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:09.995162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc17bd-0115-44a5-921e-1aff08c676bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:09.996432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc17bd-0115-44a5-921e-1aff08c676bc service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:09.997069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc17bd-0115-44a5-921e-1aff08c676bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:09.997797 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc17bd-0115-44a5-921e-1aff08c676bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:10.012534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc17bd-0115-44a5-921e-1aff08c676bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:10.015036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc17bd-0115-44a5-921e-1aff08c676bc service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:10.026099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b46b894e2ec49e882572cd50ff6bd47'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.033940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1196/2358] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:10.039515 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b46b894e2ec49e882572cd50ff6bd47'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.040207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b46b894e2ec49e882572cd50ff6bd47'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.056330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:10.064710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b46b894e2ec49e882572cd50ff6bd47'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.066573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b46b894e2ec49e882572cd50ff6bd47'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.085169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1163/2359] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:37:09 2022] DELETE /identity/v3/users/6b46b894e2ec49e882572cd50ff6bd47 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:10.089786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd43e1-783a-464d-aca2-63d99d6ebc15 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:10.090077 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd43e1-783a-464d-aca2-63d99d6ebc15 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:10.090357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd43e1-783a-464d-aca2-63d99d6ebc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:10.111060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-18dd43e1-783a-464d-aca2-63d99d6ebc15 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:10.112352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-18dd43e1-783a-464d-aca2-63d99d6ebc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:10.126915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:10.198353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1197/2360] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:10.209779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:10.211545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:10.309476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec756ab-380f-42d8-b72c-980f03f8311d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:10.313915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9a40d-84be-4a96-8764-3afbd4c26521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:10.314594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a40d-84be-4a96-8764-3afbd4c26521 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:10.319009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a40d-84be-4a96-8764-3afbd4c26521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:10.319481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a40d-84be-4a96-8764-3afbd4c26521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:10.329675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9a40d-84be-4a96-8764-3afbd4c26521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:10.331138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9a40d-84be-4a96-8764-3afbd4c26521 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:10.341154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1164/2361] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:10.683444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd43e1-783a-464d-aca2-63d99d6ebc15 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:10.686099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-18dd43e1-783a-464d-aca2-63d99d6ebc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:10.701438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd43e1-783a-464d-aca2-63d99d6ebc15 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:10.708121 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:10.708976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:10.709258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:10.709523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] PATH_INFO: `/v3/projects/8cd369e63e034e6db409c5a7a1ad5260` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:10.710686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd369e63e034e6db409c5a7a1ad5260)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:10.712613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:10.766641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd369e63e034e6db409c5a7a1ad5260', 'request_id': 'req-c6a63d5a-186d-462a-a104-35cab2416175'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.774974 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd369e63e034e6db409c5a7a1ad5260', 'request_id': 'req-c6a63d5a-186d-462a-a104-35cab2416175'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.781581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd369e63e034e6db409c5a7a1ad5260', 'request_id': 'req-c6a63d5a-186d-462a-a104-35cab2416175'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.796027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] The token cache is being invalidate because project 8cd369e63e034e6db409c5a7a1ad5260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:37:10.796349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.797025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6a63d5a-186d-462a-a104-35cab2416175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:10.798535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1198/2362] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:37:10 2022] DELETE /identity/v3/projects/8cd369e63e034e6db409c5a7a1ad5260 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:10.803846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ab6b-51d5-445e-be37-f77e42ba4046 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:10.804306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ab6b-51d5-445e-be37-f77e42ba4046 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:10.804399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ab6b-51d5-445e-be37-f77e42ba4046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:10.824471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bca9ab6b-51d5-445e-be37-f77e42ba4046 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:10.825645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bca9ab6b-51d5-445e-be37-f77e42ba4046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:10.893257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1165/2363] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:10.907204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611601-d79a-476f-99f9-635c506c00c8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:10.907737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611601-d79a-476f-99f9-635c506c00c8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:10.907830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611601-d79a-476f-99f9-635c506c00c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:10.931152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-67611601-d79a-476f-99f9-635c506c00c8 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:10.932263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-67611601-d79a-476f-99f9-635c506c00c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:10.942453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1199/2364] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:10.946928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9ab6b-51d5-445e-be37-f77e42ba4046 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:10.956945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9ab6b-51d5-445e-be37-f77e42ba4046 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:10.965766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:10.966439 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:10.966794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:10.967085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:10.967867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:37:10.968172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:10.970656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:10.987178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1166/2365] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:37:10.992850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611601-d79a-476f-99f9-635c506c00c8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:11.003221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611601-d79a-476f-99f9-635c506c00c8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:11.010958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bb55a-014e-41b6-92d2-015c2737a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:11.011556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bb55a-014e-41b6-92d2-015c2737a011 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.011835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bb55a-014e-41b6-92d2-015c2737a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.012108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bb55a-014e-41b6-92d2-015c2737a011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.012749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bb55a-014e-41b6-92d2-015c2737a011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:11.014668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bb55a-014e-41b6-92d2-015c2737a011 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:11.045159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1200/2366] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:37:10 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:11.049583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:11.059801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbbed3-a22b-46f2-a23e-4c79ec38bb4b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:11.129912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484e4a7-95fe-4629-9f26-891f240dfc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:11.130571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e4a7-95fe-4629-9f26-891f240dfc76 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.130920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e4a7-95fe-4629-9f26-891f240dfc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.131336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e4a7-95fe-4629-9f26-891f240dfc76 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.132007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484e4a7-95fe-4629-9f26-891f240dfc76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:11.133916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484e4a7-95fe-4629-9f26-891f240dfc76 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:11.159856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1167/2367] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:37:11 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:11.164304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bb55a-014e-41b6-92d2-015c2737a011 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:11.175788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bb55a-014e-41b6-92d2-015c2737a011 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:11.180810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e99e9-44bd-46c7-8795-d7fce94a9996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:11.182601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e99e9-44bd-46c7-8795-d7fce94a9996 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.182840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e99e9-44bd-46c7-8795-d7fce94a9996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.183128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e99e9-44bd-46c7-8795-d7fce94a9996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.183833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e99e9-44bd-46c7-8795-d7fce94a9996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:11.185454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e99e9-44bd-46c7-8795-d7fce94a9996 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:11.196982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1201/2368] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:11.201041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484e4a7-95fe-4629-9f26-891f240dfc76 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:11.207105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:11.222895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484e4a7-95fe-4629-9f26-891f240dfc76 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:11.233670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf8386-471d-457a-bb14-9cc8a31fb744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:11.235273 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8386-471d-457a-bb14-9cc8a31fb744 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.235562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8386-471d-457a-bb14-9cc8a31fb744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.235836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf8386-471d-457a-bb14-9cc8a31fb744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.239198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf8386-471d-457a-bb14-9cc8a31fb744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:11.241047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf8386-471d-457a-bb14-9cc8a31fb744 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:11.262670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1168/2369] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:11.267280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e99e9-44bd-46c7-8795-d7fce94a9996 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:11.282495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e99e9-44bd-46c7-8795-d7fce94a9996 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:11.288847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9a82f-3271-49bd-8145-3a5e26a34f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:11.289594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9a82f-3271-49bd-8145-3a5e26a34f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.289929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9a82f-3271-49bd-8145-3a5e26a34f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.290211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9a82f-3271-49bd-8145-3a5e26a34f67 admin admin] PATH_INFO: `/v3/projects/5314c6dd5b6e410daaab395deca85642/users/ab05c62e91dd4effa1d90026186bec91/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.309228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9a82f-3271-49bd-8145-3a5e26a34f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5314c6dd5b6e410daaab395deca85642, user_id=ab05c62e91dd4effa1d90026186bec91, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:11.312307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9a82f-3271-49bd-8145-3a5e26a34f67 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:11.338664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1202/2370] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:37:11 2022] PUT /identity/v3/projects/5314c6dd5b6e410daaab395deca85642/users/ab05c62e91dd4effa1d90026186bec91/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:11.350355 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4a546-6608-427f-9ba4-7a8c61c15c0b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.350709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4a546-6608-427f-9ba4-7a8c61c15c0b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.350922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4a546-6608-427f-9ba4-7a8c61c15c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.391984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7da4a546-6608-427f-9ba4-7a8c61c15c0b None None] MFA Rules not processed for user `ab05c62e91dd4effa1d90026186bec91`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:11.395312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7da4a546-6608-427f-9ba4-7a8c61c15c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:11.556018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f082-7d92-4f8e-b584-29e49695f84c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.556335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f082-7d92-4f8e-b584-29e49695f84c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.556624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f082-7d92-4f8e-b584-29e49695f84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.586922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3092f082-7d92-4f8e-b584-29e49695f84c None None] MFA Rules not processed for user `fffc906c79ba418cbc52e5db5ca991ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:11.588437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3092f082-7d92-4f8e-b584-29e49695f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:11.593001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1169/2371] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:11.626647 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4a546-6608-427f-9ba4-7a8c61c15c0b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:11.628162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7da4a546-6608-427f-9ba4-7a8c61c15c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:11.631852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:11.726681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1203/2372] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:11.769639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4a546-6608-427f-9ba4-7a8c61c15c0b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:11.774169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c640fd-8228-4f4f-b323-8621bd2ad1e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:11.774875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c640fd-8228-4f4f-b323-8621bd2ad1e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.775190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c640fd-8228-4f4f-b323-8621bd2ad1e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.775461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c640fd-8228-4f4f-b323-8621bd2ad1e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.790049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c640fd-8228-4f4f-b323-8621bd2ad1e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:11.793415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c640fd-8228-4f4f-b323-8621bd2ad1e2 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:11.806479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1170/2373] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:37:11.855042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092f082-7d92-4f8e-b584-29e49695f84c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:11.931280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092f082-7d92-4f8e-b584-29e49695f84c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:11.936249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:11.937116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:11.937446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:11.937786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] PATH_INFO: `/v3/users/ab05c62e91dd4effa1d90026186bec91` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:11.939112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab05c62e91dd4effa1d90026186bec91)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:11.941527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:12.012803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab05c62e91dd4effa1d90026186bec91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:12.022548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab05c62e91dd4effa1d90026186bec91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:12.023741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab05c62e91dd4effa1d90026186bec91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:12.041499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab05c62e91dd4effa1d90026186bec91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:12.042633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab05c62e91dd4effa1d90026186bec91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:12.059951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1204/2374] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:37:11 2022] DELETE /identity/v3/users/ab05c62e91dd4effa1d90026186bec91 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:12.065382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cd994-0cea-4ace-b2d7-100b648b72b7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:12.065800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cd994-0cea-4ace-b2d7-100b648b72b7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:12.066144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cd994-0cea-4ace-b2d7-100b648b72b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:12.088814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-276cd994-0cea-4ace-b2d7-100b648b72b7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:12.090635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-276cd994-0cea-4ace-b2d7-100b648b72b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:12.156125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1171/2375] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:12.168057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:12.170822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:12.244518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa2247-1e04-4c46-879e-f3cf1c15709f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:12.252553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffd8cb-7f8d-42f8-8e7f-9471041ca06b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:12.253361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffd8cb-7f8d-42f8-8e7f-9471041ca06b service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:12.255739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffd8cb-7f8d-42f8-8e7f-9471041ca06b service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:12.256181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffd8cb-7f8d-42f8-8e7f-9471041ca06b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:12.267657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffd8cb-7f8d-42f8-8e7f-9471041ca06b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:12.269123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffd8cb-7f8d-42f8-8e7f-9471041ca06b service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:12.280291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1205/2376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:12.895174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cd994-0cea-4ace-b2d7-100b648b72b7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:12.897265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-276cd994-0cea-4ace-b2d7-100b648b72b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:12.918091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cd994-0cea-4ace-b2d7-100b648b72b7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:12.933878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:12.934867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:12.935222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:12.935568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] PATH_INFO: `/v3/projects/5314c6dd5b6e410daaab395deca85642` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:12.936893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5314c6dd5b6e410daaab395deca85642)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:12.938912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:13.027581 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:13.054224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e5cf8-61d9-46bd-ae3a-c294a9cc2d76 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.054224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e5cf8-61d9-46bd-ae3a-c294a9cc2d76 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.054892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e5cf8-61d9-46bd-ae3a-c294a9cc2d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.068889 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:13.075438 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5314c6dd5b6e410daaab395deca85642', 'request_id': 'req-413a69a8-5464-4935-86a8-f3e617e28bfc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:13.076748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ee4e5cf8-61d9-46bd-ae3a-c294a9cc2d76 None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:13.078285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ee4e5cf8-61d9-46bd-ae3a-c294a9cc2d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:13.086005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5314c6dd5b6e410daaab395deca85642', 'request_id': 'req-413a69a8-5464-4935-86a8-f3e617e28bfc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:13.101229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5314c6dd5b6e410daaab395deca85642', 'request_id': 'req-413a69a8-5464-4935-86a8-f3e617e28bfc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:13.130749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] The token cache is being invalidate because project 5314c6dd5b6e410daaab395deca85642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:37:13.131216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:13.132105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-413a69a8-5464-4935-86a8-f3e617e28bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:13.134833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1172/2377] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:37:12 2022] DELETE /identity/v3/projects/5314c6dd5b6e410daaab395deca85642 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:13.144348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9989855-d224-4b12-a99b-23eb59a40ab4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.144798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9989855-d224-4b12-a99b-23eb59a40ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.145302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9989855-d224-4b12-a99b-23eb59a40ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.169223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b9989855-d224-4b12-a99b-23eb59a40ab4 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:13.170843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b9989855-d224-4b12-a99b-23eb59a40ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:13.175861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1206/2378] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:37:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:13.240645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e5cf8-61d9-46bd-ae3a-c294a9cc2d76 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:13.248728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ee4e5cf8-61d9-46bd-ae3a-c294a9cc2d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:13.288306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1173/2379] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:13.296621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ce5c1-77c1-4904-9e33-58e5079fc1bb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.296621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ce5c1-77c1-4904-9e33-58e5079fc1bb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.296829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ce5c1-77c1-4904-9e33-58e5079fc1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.320318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bc3ce5c1-77c1-4904-9e33-58e5079fc1bb None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:13.321619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc3ce5c1-77c1-4904-9e33-58e5079fc1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:13.339093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1174/2380] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:13.340277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ce5c1-77c1-4904-9e33-58e5079fc1bb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:13.352384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ce5c1-77c1-4904-9e33-58e5079fc1bb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:13.371483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:13.372360 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.372650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e5cf8-61d9-46bd-ae3a-c294a9cc2d76 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:13.374102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.375097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.376602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:37:13.376908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:13.378543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:13.391036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469e526-1737-4ce2-b25f-230150ec4815 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:13.392156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469e526-1737-4ce2-b25f-230150ec4815 service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.392429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469e526-1737-4ce2-b25f-230150ec4815 service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.392773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469e526-1737-4ce2-b25f-230150ec4815 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.419189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1175/2381] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:37:13.420567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469e526-1737-4ce2-b25f-230150ec4815 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:13.422318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469e526-1737-4ce2-b25f-230150ec4815 service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:13.439101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:13.446347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1207/2382] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:37:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:37:13.473104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cea959-a3e6-4dee-85c0-1d29b0a25e51 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:13.489716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d72f1-ab87-4a65-8431-f0f29ca4e1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:13.490963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d72f1-ab87-4a65-8431-f0f29ca4e1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.490963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d72f1-ab87-4a65-8431-f0f29ca4e1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.492398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d72f1-ab87-4a65-8431-f0f29ca4e1c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.493266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d72f1-ab87-4a65-8431-f0f29ca4e1c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:13.495978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d72f1-ab87-4a65-8431-f0f29ca4e1c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:13.613769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1176/2383] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:37:13 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:13.615975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469e526-1737-4ce2-b25f-230150ec4815 service swift] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:13.632472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469e526-1737-4ce2-b25f-230150ec4815 service swift] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:13.782756 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9f9bc-88de-4e02-a4df-a04afedeaaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:13.783398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9f9bc-88de-4e02-a4df-a04afedeaaac admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.783680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9f9bc-88de-4e02-a4df-a04afedeaaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.783992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9f9bc-88de-4e02-a4df-a04afedeaaac admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.785188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9f9bc-88de-4e02-a4df-a04afedeaaac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:13.788754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9f9bc-88de-4e02-a4df-a04afedeaaac admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:13.828007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1208/2384] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:37:13 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:37:13.844258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d72f1-ab87-4a65-8431-f0f29ca4e1c6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:13.867540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d72f1-ab87-4a65-8431-f0f29ca4e1c6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:13.883646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a902dd-75de-49ac-8bcd-2c61c36fba95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:13.884412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a902dd-75de-49ac-8bcd-2c61c36fba95 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.884786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a902dd-75de-49ac-8bcd-2c61c36fba95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.885232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a902dd-75de-49ac-8bcd-2c61c36fba95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.886010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a902dd-75de-49ac-8bcd-2c61c36fba95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:13.891282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a902dd-75de-49ac-8bcd-2c61c36fba95 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:13.906457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1177/2385] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:13.910468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9f9bc-88de-4e02-a4df-a04afedeaaac admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:13.925748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9f9bc-88de-4e02-a4df-a04afedeaaac admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:13.931847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb54fa7-736d-4fa1-9e27-5033014643c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:13.932564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54fa7-736d-4fa1-9e27-5033014643c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:13.932987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54fa7-736d-4fa1-9e27-5033014643c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:13.933451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54fa7-736d-4fa1-9e27-5033014643c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:13.934339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb54fa7-736d-4fa1-9e27-5033014643c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:13.936207 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb54fa7-736d-4fa1-9e27-5033014643c7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:13.961087 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1209/2386] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:13.962487 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a902dd-75de-49ac-8bcd-2c61c36fba95 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:13.996330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a902dd-75de-49ac-8bcd-2c61c36fba95 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:14.002769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:14.003559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:14.004099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:14.004516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] PATH_INFO: `/v3/projects/b38ac6340946490db53227ccf2423181/users/17a898436e444ea583e8f28da561c49d/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:14.030135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38ac6340946490db53227ccf2423181, user_id=17a898436e444ea583e8f28da561c49d, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:14.033342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:14.072479 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1178/2387] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:37:13 2022] PUT /identity/v3/projects/b38ac6340946490db53227ccf2423181/users/17a898436e444ea583e8f28da561c49d/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:14.083973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a892a89-da4f-4fd2-92ac-3f0e1821a9e7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:14.083973 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a892a89-da4f-4fd2-92ac-3f0e1821a9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:14.085003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a892a89-da4f-4fd2-92ac-3f0e1821a9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:14.158539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2a892a89-da4f-4fd2-92ac-3f0e1821a9e7 None None] MFA Rules not processed for user `17a898436e444ea583e8f28da561c49d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:14.161146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2a892a89-da4f-4fd2-92ac-3f0e1821a9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:14.174586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:14.282817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1210/2388] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:14.294880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:14.297635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:14.344590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae102-2daa-4af6-a6e4-8401ad4facdc None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:14.345051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae102-2daa-4af6-a6e4-8401ad4facdc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:14.345419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae102-2daa-4af6-a6e4-8401ad4facdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:14.370816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-36dae102-2daa-4af6-a6e4-8401ad4facdc None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:14.372487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-36dae102-2daa-4af6-a6e4-8401ad4facdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:14.385343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:14.415473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547250ea-8c9f-4710-9cbd-65e2b338b697 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:14.421345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb59a1a-c0d2-4713-8565-cf67b0fcd5eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:14.421976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb59a1a-c0d2-4713-8565-cf67b0fcd5eb service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:14.422257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb59a1a-c0d2-4713-8565-cf67b0fcd5eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:14.422525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb59a1a-c0d2-4713-8565-cf67b0fcd5eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:14.432397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb59a1a-c0d2-4713-8565-cf67b0fcd5eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:14.434101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb59a1a-c0d2-4713-8565-cf67b0fcd5eb service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:14.445602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1179/2389] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:14.454604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1211/2390] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 31 04:37:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:14.471936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb59a1a-c0d2-4713-8565-cf67b0fcd5eb service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:14.560183 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb59a1a-c0d2-4713-8565-cf67b0fcd5eb service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:14.565060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7581e8-3449-464c-81aa-10f3f09b7787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:14.565661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7581e8-3449-464c-81aa-10f3f09b7787 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:14.565935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7581e8-3449-464c-81aa-10f3f09b7787 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:14.566250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7581e8-3449-464c-81aa-10f3f09b7787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:14.574423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7581e8-3449-464c-81aa-10f3f09b7787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:14.576023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7581e8-3449-464c-81aa-10f3f09b7787 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:14.585261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1180/2391] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:37:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 04:37:14.984638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e8bd-6c52-4425-b0b3-0effc468009f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:14.985794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e8bd-6c52-4425-b0b3-0effc468009f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:14.986177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e8bd-6c52-4425-b0b3-0effc468009f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:15.018858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3c69e8bd-6c52-4425-b0b3-0effc468009f None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:15.024903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3c69e8bd-6c52-4425-b0b3-0effc468009f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:15.033694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1212/2392] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:37:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:15.049349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7581e8-3449-464c-81aa-10f3f09b7787 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:15.065159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7581e8-3449-464c-81aa-10f3f09b7787 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:15.073359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86efbe6-ad60-4e5f-af18-74f0aaec939f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:15.074078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86efbe6-ad60-4e5f-af18-74f0aaec939f service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:15.074363 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86efbe6-ad60-4e5f-af18-74f0aaec939f service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:15.074638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86efbe6-ad60-4e5f-af18-74f0aaec939f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:15.085251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86efbe6-ad60-4e5f-af18-74f0aaec939f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:15.086940 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86efbe6-ad60-4e5f-af18-74f0aaec939f service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:15.095665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1181/2393] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:37:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 04:37:15.548992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38845d-f47a-4bff-ad87-b426e54276d3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:15.551570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38845d-f47a-4bff-ad87-b426e54276d3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:15.553139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38845d-f47a-4bff-ad87-b426e54276d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:15.581008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-dd38845d-f47a-4bff-ad87-b426e54276d3 None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:15.585193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dd38845d-f47a-4bff-ad87-b426e54276d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:15.612948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1213/2394] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 31 04:37:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:15.623026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86efbe6-ad60-4e5f-af18-74f0aaec939f service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:15.645427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86efbe6-ad60-4e5f-af18-74f0aaec939f service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:15.653658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef48d4-c5a2-4eeb-9db1-790d78742533 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:15.654559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef48d4-c5a2-4eeb-9db1-790d78742533 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:15.655024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef48d4-c5a2-4eeb-9db1-790d78742533 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:15.655935 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef48d4-c5a2-4eeb-9db1-790d78742533 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:15.667859 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef48d4-c5a2-4eeb-9db1-790d78742533 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:15.669841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef48d4-c5a2-4eeb-9db1-790d78742533 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:15.680128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1182/2395] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:37:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:37:18.164663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38845d-f47a-4bff-ad87-b426e54276d3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:18.166742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dd38845d-f47a-4bff-ad87-b426e54276d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:18.246765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38845d-f47a-4bff-ad87-b426e54276d3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:18.251318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:18.252266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:18.252699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:18.252968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] PATH_INFO: `/v3/users/294d5d11423c4d2bad05afacdd999995` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:18.254276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=294d5d11423c4d2bad05afacdd999995)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:18.257403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:18.321430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:18.325130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '294d5d11423c4d2bad05afacdd999995'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.340476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '294d5d11423c4d2bad05afacdd999995'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.341872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '294d5d11423c4d2bad05afacdd999995'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.356308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '294d5d11423c4d2bad05afacdd999995'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.357109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '294d5d11423c4d2bad05afacdd999995'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.372557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1214/2396] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:37:18 2022] DELETE /identity/v3/users/294d5d11423c4d2bad05afacdd999995 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:18.377165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24187f76-046f-43fa-b16b-db5976924baa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:18.377653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24187f76-046f-43fa-b16b-db5976924baa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:18.377749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24187f76-046f-43fa-b16b-db5976924baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:18.396812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-24187f76-046f-43fa-b16b-db5976924baa None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:18.397998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24187f76-046f-43fa-b16b-db5976924baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:18.471753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1183/2397] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:18.492924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:18.494972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:18.564120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07890e1-b3bf-4104-aec3-b28fe9db3bff admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:18.567880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea7519-a744-418c-a158-5c548801f68f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:18.568652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7519-a744-418c-a158-5c548801f68f service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:18.569150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7519-a744-418c-a158-5c548801f68f service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:18.569878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7519-a744-418c-a158-5c548801f68f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:18.579244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea7519-a744-418c-a158-5c548801f68f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:18.580736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea7519-a744-418c-a158-5c548801f68f service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:18.591244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1215/2398] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:18.769894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24187f76-046f-43fa-b16b-db5976924baa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:18.771188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24187f76-046f-43fa-b16b-db5976924baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:18.786410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24187f76-046f-43fa-b16b-db5976924baa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:18.792356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:18.793179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:18.793645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:18.794115 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] PATH_INFO: `/v3/projects/80dd3f85755d4deea3fc0192c1cbbc4b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:18.795543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80dd3f85755d4deea3fc0192c1cbbc4b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:18.798248 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:18.836235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:18.849416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80dd3f85755d4deea3fc0192c1cbbc4b', 'request_id': 'req-e5020222-d3b6-4abe-8f80-04294a049a68'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.855896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80dd3f85755d4deea3fc0192c1cbbc4b', 'request_id': 'req-e5020222-d3b6-4abe-8f80-04294a049a68'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.864523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80dd3f85755d4deea3fc0192c1cbbc4b', 'request_id': 'req-e5020222-d3b6-4abe-8f80-04294a049a68'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.876210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] The token cache is being invalidate because project 80dd3f85755d4deea3fc0192c1cbbc4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:37:18.876210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.876454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5020222-d3b6-4abe-8f80-04294a049a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:18.878050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1184/2399] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:37:18 2022] DELETE /identity/v3/projects/80dd3f85755d4deea3fc0192c1cbbc4b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:37:18.883648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20b4d4-963d-4532-aab1-a58dbb6be2b1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:18.883776 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20b4d4-963d-4532-aab1-a58dbb6be2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:18.884035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20b4d4-963d-4532-aab1-a58dbb6be2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:18.912072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8e20b4d4-963d-4532-aab1-a58dbb6be2b1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:18.913368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8e20b4d4-963d-4532-aab1-a58dbb6be2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:19.030235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1216/2400] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:19.102349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e6802-265b-4c9d-9e5c-e43337a8edc1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:19.102642 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e6802-265b-4c9d-9e5c-e43337a8edc1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:19.102950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e6802-265b-4c9d-9e5c-e43337a8edc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:19.132529 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b68e6802-265b-4c9d-9e5c-e43337a8edc1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:19.134422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b68e6802-265b-4c9d-9e5c-e43337a8edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:19.142811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1185/2401] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:19.147239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20b4d4-963d-4532-aab1-a58dbb6be2b1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:19.157774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20b4d4-963d-4532-aab1-a58dbb6be2b1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:19.162613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:19.163481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:19.163839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:19.164268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:19.164996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:37:19.165571 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:19.167157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:19.179263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1217/2402] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:37:19.183680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e6802-265b-4c9d-9e5c-e43337a8edc1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:19.193912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e6802-265b-4c9d-9e5c-e43337a8edc1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:19.202059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da0c80-d76d-4cb3-9b42-e8d3a7f6bd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:19.202708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da0c80-d76d-4cb3-9b42-e8d3a7f6bd7d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:19.203015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da0c80-d76d-4cb3-9b42-e8d3a7f6bd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:19.203289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da0c80-d76d-4cb3-9b42-e8d3a7f6bd7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:19.203937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da0c80-d76d-4cb3-9b42-e8d3a7f6bd7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:19.205875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da0c80-d76d-4cb3-9b42-e8d3a7f6bd7d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:19.237674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1186/2403] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:37:19 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:19.242672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:19.252912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da1d3f-2a87-4eb5-9358-f9ce738a745c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:19.311286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:19.326430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1613ff9-92e1-40ea-94f5-2e0df56664ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:19.327223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613ff9-92e1-40ea-94f5-2e0df56664ac admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:19.327795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613ff9-92e1-40ea-94f5-2e0df56664ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:19.328146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613ff9-92e1-40ea-94f5-2e0df56664ac admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:19.329099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1613ff9-92e1-40ea-94f5-2e0df56664ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:19.332599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1613ff9-92e1-40ea-94f5-2e0df56664ac admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:19.378813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1218/2404] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:37:19 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:19.379681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da0c80-d76d-4cb3-9b42-e8d3a7f6bd7d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:19.396603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da0c80-d76d-4cb3-9b42-e8d3a7f6bd7d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:19.402617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d623376-4dcf-4f6b-9f61-fab651efe49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:19.403406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d623376-4dcf-4f6b-9f61-fab651efe49a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:19.403752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d623376-4dcf-4f6b-9f61-fab651efe49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:19.404088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d623376-4dcf-4f6b-9f61-fab651efe49a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:19.404839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d623376-4dcf-4f6b-9f61-fab651efe49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:19.406594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d623376-4dcf-4f6b-9f61-fab651efe49a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:19.418543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1187/2405] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:19.423012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1613ff9-92e1-40ea-94f5-2e0df56664ac admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:19.435073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1613ff9-92e1-40ea-94f5-2e0df56664ac admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:19.446754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae342b4-b91a-4536-88bb-d4926577c3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:19.448624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae342b4-b91a-4536-88bb-d4926577c3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:19.449098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae342b4-b91a-4536-88bb-d4926577c3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:19.450849 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae342b4-b91a-4536-88bb-d4926577c3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:19.451900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae342b4-b91a-4536-88bb-d4926577c3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:19.455588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae342b4-b91a-4536-88bb-d4926577c3c0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:19.476709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1219/2406] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:37:19.477702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d623376-4dcf-4f6b-9f61-fab651efe49a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:19.495325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d623376-4dcf-4f6b-9f61-fab651efe49a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:19.500350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:19.501640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:19.502316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:19.502777 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] PATH_INFO: `/v3/projects/6714373199f5419a9a70267187067934/users/6b1139bf460a49d2b0156912be60fa36/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:19.523003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6714373199f5419a9a70267187067934, user_id=6b1139bf460a49d2b0156912be60fa36, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:19.526369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:19.583356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1188/2407] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:37:19 2022] PUT /identity/v3/projects/6714373199f5419a9a70267187067934/users/6b1139bf460a49d2b0156912be60fa36/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:19.589555 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201ea9e-243f-4f20-9693-e95ac7f07723 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:19.589841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201ea9e-243f-4f20-9693-e95ac7f07723 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:19.590115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201ea9e-243f-4f20-9693-e95ac7f07723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:19.619619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8201ea9e-243f-4f20-9693-e95ac7f07723 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:19.621144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8201ea9e-243f-4f20-9693-e95ac7f07723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:19.780819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1220/2408] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:19.789496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:19.791497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:19.968698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ee1be-e84f-4bbc-8b06-88871863acf6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:20.012516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:20.013155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:20.013772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:20.014053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:20.032192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:20.033779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:20.060692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1189/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:20.161934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22920d1-a8a5-4f44-a981-c8cd23953c4a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:20.162236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22920d1-a8a5-4f44-a981-c8cd23953c4a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:20.162528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22920d1-a8a5-4f44-a981-c8cd23953c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:20.189989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f22920d1-a8a5-4f44-a981-c8cd23953c4a None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:20.191839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f22920d1-a8a5-4f44-a981-c8cd23953c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:20.288688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1221/2410] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:37:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:20.305445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:20.309703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:20.411425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:20.421035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b37fe-c742-4776-9c94-b3fd7c63b987 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:20.454423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774832a8-b67f-4010-b1c8-f02cca53d36a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:20.463594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774832a8-b67f-4010-b1c8-f02cca53d36a service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:20.464194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774832a8-b67f-4010-b1c8-f02cca53d36a service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:20.466300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774832a8-b67f-4010-b1c8-f02cca53d36a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:20.493527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774832a8-b67f-4010-b1c8-f02cca53d36a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:20.496032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774832a8-b67f-4010-b1c8-f02cca53d36a service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:20.519523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1190/2411] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:37:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:37:28.827517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764cb23-fe68-46e4-ab6f-9f2fa02edd46 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:28.828122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764cb23-fe68-46e4-ab6f-9f2fa02edd46 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:28.828455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764cb23-fe68-46e4-ab6f-9f2fa02edd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:28.911786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7764cb23-fe68-46e4-ab6f-9f2fa02edd46 None None] MFA Rules not processed for user `6b1139bf460a49d2b0156912be60fa36`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:28.913107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7764cb23-fe68-46e4-ab6f-9f2fa02edd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:29.041729 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1222/2412] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:29.046403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774832a8-b67f-4010-b1c8-f02cca53d36a service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:29.089787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774832a8-b67f-4010-b1c8-f02cca53d36a service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:29.105662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a319b1-d036-4a21-a57f-2264485504f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:29.109917 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a319b1-d036-4a21-a57f-2264485504f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:29.119966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a319b1-d036-4a21-a57f-2264485504f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:29.122304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a319b1-d036-4a21-a57f-2264485504f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:29.123230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a319b1-d036-4a21-a57f-2264485504f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:29.137349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a319b1-d036-4a21-a57f-2264485504f1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:29.217299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1191/2413] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:37:29 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:29.221150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764cb23-fe68-46e4-ab6f-9f2fa02edd46 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:29.230929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:29.243838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764cb23-fe68-46e4-ab6f-9f2fa02edd46 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:29.450512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45103123-6950-4625-8cdf-7151e2cfccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:29.451448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45103123-6950-4625-8cdf-7151e2cfccba admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:29.456858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45103123-6950-4625-8cdf-7151e2cfccba admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:29.460085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45103123-6950-4625-8cdf-7151e2cfccba admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:29.463229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45103123-6950-4625-8cdf-7151e2cfccba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:29.473755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45103123-6950-4625-8cdf-7151e2cfccba admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:29.564082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a319b1-d036-4a21-a57f-2264485504f1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:29.566810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1223/2414] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:37:29 2022] POST /identity/v3/users => generated 299 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:29.583276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a319b1-d036-4a21-a57f-2264485504f1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:29.605749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3f892-3cb2-42cc-8bb6-288cbaa83418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:29.607988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f892-3cb2-42cc-8bb6-288cbaa83418 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:29.608392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f892-3cb2-42cc-8bb6-288cbaa83418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:29.608667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f892-3cb2-42cc-8bb6-288cbaa83418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:29.609539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3f892-3cb2-42cc-8bb6-288cbaa83418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:29.611666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3f892-3cb2-42cc-8bb6-288cbaa83418 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:29.642582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1192/2415] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:29.646472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45103123-6950-4625-8cdf-7151e2cfccba admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:29.661906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45103123-6950-4625-8cdf-7151e2cfccba admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:29.687233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cd5e9-4c98-4a0f-99fc-391e31870be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:29.687926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd5e9-4c98-4a0f-99fc-391e31870be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:29.688205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd5e9-4c98-4a0f-99fc-391e31870be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:29.688475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd5e9-4c98-4a0f-99fc-391e31870be9 admin admin] PATH_INFO: `/v3/projects/f992ae8c1f3b47dd88bd3f2b20216125/users/5ddb77df65c345409f592ad86032f84c/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:29.723953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cd5e9-4c98-4a0f-99fc-391e31870be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f992ae8c1f3b47dd88bd3f2b20216125, user_id=5ddb77df65c345409f592ad86032f84c, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:29.727636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cd5e9-4c98-4a0f-99fc-391e31870be9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:29.763379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1224/2416] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:37:29 2022] PUT /identity/v3/projects/f992ae8c1f3b47dd88bd3f2b20216125/users/5ddb77df65c345409f592ad86032f84c/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:29.767566 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3f892-3cb2-42cc-8bb6-288cbaa83418 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:29.779661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3f892-3cb2-42cc-8bb6-288cbaa83418 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:29.967783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:29.968449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:29.968729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:29.968988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:29.969672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:29.971505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:30.001853 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1193/2417] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:30.005866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cd5e9-4c98-4a0f-99fc-391e31870be9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:30.027175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cd5e9-4c98-4a0f-99fc-391e31870be9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:30.041860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e259b0-8909-4acf-b5e4-8aba0e84d507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:30.042785 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e259b0-8909-4acf-b5e4-8aba0e84d507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:30.043213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e259b0-8909-4acf-b5e4-8aba0e84d507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:30.043656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e259b0-8909-4acf-b5e4-8aba0e84d507 admin admin] PATH_INFO: `/v3/projects/f992ae8c1f3b47dd88bd3f2b20216125/users/5ddb77df65c345409f592ad86032f84c/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:30.046039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e259b0-8909-4acf-b5e4-8aba0e84d507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f992ae8c1f3b47dd88bd3f2b20216125, user_id=5ddb77df65c345409f592ad86032f84c, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:30.049330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e259b0-8909-4acf-b5e4-8aba0e84d507 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:30.086350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1225/2418] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:37:30 2022] PUT /identity/v3/projects/f992ae8c1f3b47dd88bd3f2b20216125/users/5ddb77df65c345409f592ad86032f84c/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:30.551727 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:30.554104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:30.588952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f1acb-0368-4222-b419-e51d58c3fe56 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:30.782280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:30.783282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:30.783694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:30.783970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] PATH_INFO: `/v3/users/17a898436e444ea583e8f28da561c49d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:30.786092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a898436e444ea583e8f28da561c49d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:30.789838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:30.924504 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a898436e444ea583e8f28da561c49d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:30.948074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a898436e444ea583e8f28da561c49d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:30.948693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a898436e444ea583e8f28da561c49d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:30.950394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:31.030341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a898436e444ea583e8f28da561c49d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:31.031549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a898436e444ea583e8f28da561c49d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:31.079317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1194/2419] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:37:30 2022] DELETE /identity/v3/users/17a898436e444ea583e8f28da561c49d => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:31.097517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73876c-89f4-451b-95af-d75d65f3baaf None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:31.097981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73876c-89f4-451b-95af-d75d65f3baaf None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:31.098252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73876c-89f4-451b-95af-d75d65f3baaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:31.154350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2d73876c-89f4-451b-95af-d75d65f3baaf None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:31.155092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2d73876c-89f4-451b-95af-d75d65f3baaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:31.368222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1226/2420] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:31.447603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:31.449803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:31.557495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83325b6e-dc51-476d-b5fd-1a396415a604 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:31.563004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28789af3-fa98-4c99-90f9-79ba17658953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:31.563982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789af3-fa98-4c99-90f9-79ba17658953 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:31.564468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789af3-fa98-4c99-90f9-79ba17658953 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:31.564957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789af3-fa98-4c99-90f9-79ba17658953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:31.579408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28789af3-fa98-4c99-90f9-79ba17658953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:31.582435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28789af3-fa98-4c99-90f9-79ba17658953 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:31.597865 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1195/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:33.139812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73876c-89f4-451b-95af-d75d65f3baaf None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:33.142154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2d73876c-89f4-451b-95af-d75d65f3baaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:33.162107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73876c-89f4-451b-95af-d75d65f3baaf None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:33.175737 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:33.176767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:33.177147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:33.178545 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] PATH_INFO: `/v3/projects/b38ac6340946490db53227ccf2423181` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:33.181810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38ac6340946490db53227ccf2423181)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:33.188032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:33.358678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38ac6340946490db53227ccf2423181', 'request_id': 'req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:33.364493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:33.423381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38ac6340946490db53227ccf2423181', 'request_id': 'req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:33.545182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38ac6340946490db53227ccf2423181', 'request_id': 'req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:33.585721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] The token cache is being invalidate because project b38ac6340946490db53227ccf2423181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:37:33.586338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:33.587268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:33.589804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1227/2422] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:37:33 2022] DELETE /identity/v3/projects/b38ac6340946490db53227ccf2423181 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:39.454114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0438941-339a-4b81-8eb7-02644d6c2326 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:39.454638 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0438941-339a-4b81-8eb7-02644d6c2326 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:39.454879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0438941-339a-4b81-8eb7-02644d6c2326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:39.548792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e0438941-339a-4b81-8eb7-02644d6c2326 None None] MFA Rules not processed for user `5ddb77df65c345409f592ad86032f84c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:39.552045 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0438941-339a-4b81-8eb7-02644d6c2326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:39.704494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1196/2423] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:39.726770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:39.726770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:39.883145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac7ead-71a8-4dd1-843d-fd8c2b619a28 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:39.897769 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:39.898735 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:39.899190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:39.899630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:39.902688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:39.996044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:39.997954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:40.010337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1228/2424] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:40.251808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d47b-0ffc-44b9-8d1d-4d6ad40df2f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:40.252840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d47b-0ffc-44b9-8d1d-4d6ad40df2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:40.253268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d47b-0ffc-44b9-8d1d-4d6ad40df2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:40.294984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c67d47b-0ffc-44b9-8d1d-4d6ad40df2f0 None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:40.297579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c67d47b-0ffc-44b9-8d1d-4d6ad40df2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:40.379000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1197/2425] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:37:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:40.398250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:40.400273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:40.513338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1767a6b-5979-4b12-a85d-70e09dfe2b53 service glance] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:40.531312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f04c38-159b-4ff1-96e2-632d897b9058 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:40.539743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f04c38-159b-4ff1-96e2-632d897b9058 service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:40.539743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f04c38-159b-4ff1-96e2-632d897b9058 service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:40.539743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f04c38-159b-4ff1-96e2-632d897b9058 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:40.555959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f04c38-159b-4ff1-96e2-632d897b9058 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:40.557740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f04c38-159b-4ff1-96e2-632d897b9058 service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:40.571738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1229/2426] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:37:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 04:37:41.418867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67d47b-0ffc-44b9-8d1d-4d6ad40df2f0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:41.421476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c67d47b-0ffc-44b9-8d1d-4d6ad40df2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:41.446493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:41.677982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67d47b-0ffc-44b9-8d1d-4d6ad40df2f0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:41.700307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:41.700983 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:41.701268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:41.701539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] PATH_INFO: `/v3/users/e5541503f3834866aa1d4801fefda98d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:41.703556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5541503f3834866aa1d4801fefda98d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:41.705585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:41.850682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5541503f3834866aa1d4801fefda98d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:41.868613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5541503f3834866aa1d4801fefda98d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:41.869292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5541503f3834866aa1d4801fefda98d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:41.902033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5541503f3834866aa1d4801fefda98d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:41.903301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5541503f3834866aa1d4801fefda98d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:41.965922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1198/2427] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:37:41 2022] DELETE /identity/v3/users/e5541503f3834866aa1d4801fefda98d => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:41.971059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e255a0f-39ec-439d-b684-8e6e933b896b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:41.971627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e255a0f-39ec-439d-b684-8e6e933b896b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:41.971944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e255a0f-39ec-439d-b684-8e6e933b896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:42.015637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5e255a0f-39ec-439d-b684-8e6e933b896b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:42.017534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5e255a0f-39ec-439d-b684-8e6e933b896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:42.047822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:42.226228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:42.228065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:42.228548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1230/2428] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:42.371505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa6b6b-9ef9-40ce-a768-466e5e433b6c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:42.376117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9a495-4b91-4f21-b967-ccf0d4ca3d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:42.376991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9a495-4b91-4f21-b967-ccf0d4ca3d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:42.377476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9a495-4b91-4f21-b967-ccf0d4ca3d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:42.377923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9a495-4b91-4f21-b967-ccf0d4ca3d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:42.394149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9a495-4b91-4f21-b967-ccf0d4ca3d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:42.395786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9a495-4b91-4f21-b967-ccf0d4ca3d22 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:42.408942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1199/2429] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:37:42.869593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e255a0f-39ec-439d-b684-8e6e933b896b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:42.871162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5e255a0f-39ec-439d-b684-8e6e933b896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:42.892655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e255a0f-39ec-439d-b684-8e6e933b896b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:42.936331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:42.939503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:42.942102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:42.944573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] PATH_INFO: `/v3/projects/c707ecbc947f49debcd248a815c69f41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:42.948969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c707ecbc947f49debcd248a815c69f41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:42.954381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:43.052724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c707ecbc947f49debcd248a815c69f41', 'request_id': 'req-c3a846e5-7049-4beb-88a7-066d3057dc9d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:43.073931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c707ecbc947f49debcd248a815c69f41', 'request_id': 'req-c3a846e5-7049-4beb-88a7-066d3057dc9d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:43.085143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c707ecbc947f49debcd248a815c69f41', 'request_id': 'req-c3a846e5-7049-4beb-88a7-066d3057dc9d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:43.097670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] The token cache is being invalidate because project c707ecbc947f49debcd248a815c69f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:37:43.097971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:43.098648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a846e5-7049-4beb-88a7-066d3057dc9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:43.101151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1231/2430] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:37:42 2022] DELETE /identity/v3/projects/c707ecbc947f49debcd248a815c69f41 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:43.106585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfb41d-5689-471a-bcbd-9e5d24b12c3b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:43.107445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfb41d-5689-471a-bcbd-9e5d24b12c3b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:43.107861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfb41d-5689-471a-bcbd-9e5d24b12c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:43.151141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-81dfb41d-5689-471a-bcbd-9e5d24b12c3b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:43.152396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-81dfb41d-5689-471a-bcbd-9e5d24b12c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:43.179827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:43.260331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1200/2431] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:43.263323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d844f4c-a039-4c83-a71b-9c27b61551a9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:43.263932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d844f4c-a039-4c83-a71b-9c27b61551a9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:43.264358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d844f4c-a039-4c83-a71b-9c27b61551a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:43.290178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7d844f4c-a039-4c83-a71b-9c27b61551a9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:43.291473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7d844f4c-a039-4c83-a71b-9c27b61551a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:43.306193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1232/2432] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:43.307610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfb41d-5689-471a-bcbd-9e5d24b12c3b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:43.325851 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfb41d-5689-471a-bcbd-9e5d24b12c3b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:43.347974 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:43.348798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:43.349306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:43.349943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:43.351138 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:37:43.351645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:43.353654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:43.368270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1201/2433] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:37:43.372717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d844f4c-a039-4c83-a71b-9c27b61551a9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:43.387711 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d844f4c-a039-4c83-a71b-9c27b61551a9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:43.404387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5a04b-7c73-488a-92b8-69307460a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:43.405201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5a04b-7c73-488a-92b8-69307460a293 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:43.405569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5a04b-7c73-488a-92b8-69307460a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:43.405899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5a04b-7c73-488a-92b8-69307460a293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:43.406675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5a04b-7c73-488a-92b8-69307460a293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:43.408853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5a04b-7c73-488a-92b8-69307460a293 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:43.470172 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1233/2434] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:37:43 2022] POST /identity/v3/projects => generated 397 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:37:43.474338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:43.498997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fac59-ea14-405d-b9f8-913cff08b3f0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:43.640579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395c719-7343-475f-abc7-e86ab6451e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:43.641419 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c719-7343-475f-abc7-e86ab6451e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:43.641685 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c719-7343-475f-abc7-e86ab6451e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:43.642050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c719-7343-475f-abc7-e86ab6451e09 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:43.642803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395c719-7343-475f-abc7-e86ab6451e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:43.660480 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395c719-7343-475f-abc7-e86ab6451e09 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:43.706376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1202/2435] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:37:43 2022] POST /identity/v3/users => generated 299 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:37:43.711268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5a04b-7c73-488a-92b8-69307460a293 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:43.726858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5a04b-7c73-488a-92b8-69307460a293 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:43.741527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d008433-6824-4cfa-aeea-82926406a1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:43.742321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d008433-6824-4cfa-aeea-82926406a1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:43.742746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d008433-6824-4cfa-aeea-82926406a1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:43.743029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d008433-6824-4cfa-aeea-82926406a1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:43.744123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d008433-6824-4cfa-aeea-82926406a1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:43.759713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d008433-6824-4cfa-aeea-82926406a1f0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:43.801977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395c719-7343-475f-abc7-e86ab6451e09 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:43.802964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1234/2436] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:37:43.832344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395c719-7343-475f-abc7-e86ab6451e09 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:43.856030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff97ef-e210-4cce-b994-1fed3f6ce178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:43.856712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff97ef-e210-4cce-b994-1fed3f6ce178 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:43.856992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff97ef-e210-4cce-b994-1fed3f6ce178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:43.857262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff97ef-e210-4cce-b994-1fed3f6ce178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:43.858155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff97ef-e210-4cce-b994-1fed3f6ce178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:43.864970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff97ef-e210-4cce-b994-1fed3f6ce178 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:43.914361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1203/2437] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:37:43.918539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d008433-6824-4cfa-aeea-82926406a1f0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:43.939911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d008433-6824-4cfa-aeea-82926406a1f0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:43.954666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:43.955601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:43.955931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:43.956209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] PATH_INFO: `/v3/projects/20cd24e91c574f60b16950db42a18b30/users/8430fd71c41b46fd944013d15060d3a7/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:43.983643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20cd24e91c574f60b16950db42a18b30, user_id=8430fd71c41b46fd944013d15060d3a7, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:43.986751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:44.019474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1235/2438] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:37:43 2022] PUT /identity/v3/projects/20cd24e91c574f60b16950db42a18b30/users/8430fd71c41b46fd944013d15060d3a7/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:44.101649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5687e93-d168-47b3-99d7-633eb9c5e412 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:44.102139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5687e93-d168-47b3-99d7-633eb9c5e412 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:44.102452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5687e93-d168-47b3-99d7-633eb9c5e412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:44.153378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c5687e93-d168-47b3-99d7-633eb9c5e412 None None] MFA Rules not processed for user `8430fd71c41b46fd944013d15060d3a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:44.157253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c5687e93-d168-47b3-99d7-633eb9c5e412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:44.370511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1204/2439] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:44.387476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:44.392625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:44.512747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:44.598416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1f077-d3f6-4592-ba46-e59a14f802f2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:44.661469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb11f7-2886-4dc3-8c3a-a5879634d2d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:44.662139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb11f7-2886-4dc3-8c3a-a5879634d2d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:44.662416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb11f7-2886-4dc3-8c3a-a5879634d2d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:44.662792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb11f7-2886-4dc3-8c3a-a5879634d2d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:44.679131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb11f7-2886-4dc3-8c3a-a5879634d2d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:44.680740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb11f7-2886-4dc3-8c3a-a5879634d2d5 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:44.699911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1236/2440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:44 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:45.020079 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b693-ac0a-43fd-bcce-8d294e92580f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:45.020372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b693-ac0a-43fd-bcce-8d294e92580f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:45.020648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b693-ac0a-43fd-bcce-8d294e92580f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:45.049101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4984b693-ac0a-43fd-bcce-8d294e92580f None None] MFA Rules not processed for user `fffc906c79ba418cbc52e5db5ca991ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:45.050345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4984b693-ac0a-43fd-bcce-8d294e92580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:45.096267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:45.172391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1205/2441] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:48.214766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fcfc2-e5b0-41dc-9eeb-133aa4c2c554 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.215641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fcfc2-e5b0-41dc-9eeb-133aa4c2c554 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.215926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fcfc2-e5b0-41dc-9eeb-133aa4c2c554 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:48.218486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1237/2442] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:37:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:37:48.231945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd927a-faef-4ad4-aa26-b438aeefce73 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.232584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd927a-faef-4ad4-aa26-b438aeefce73 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.232584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd927a-faef-4ad4-aa26-b438aeefce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:48.257153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5edd927a-faef-4ad4-aa26-b438aeefce73 None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:48.261167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5edd927a-faef-4ad4-aa26-b438aeefce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:48.366585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1206/2443] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:37:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:48.378924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fcfc2-e5b0-41dc-9eeb-133aa4c2c554 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:48.380681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ba0fcfc2-e5b0-41dc-9eeb-133aa4c2c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:48.456494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fcfc2-e5b0-41dc-9eeb-133aa4c2c554 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:48.461813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d468985-25e9-4c28-ad56-5bad8ff8066a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:48.461813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d468985-25e9-4c28-ad56-5bad8ff8066a service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.461813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d468985-25e9-4c28-ad56-5bad8ff8066a service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.461813 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d468985-25e9-4c28-ad56-5bad8ff8066a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:48.475731 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d468985-25e9-4c28-ad56-5bad8ff8066a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:48.477861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d468985-25e9-4c28-ad56-5bad8ff8066a service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:48.490972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1238/2444] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:48.568034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bdc38-7922-4d7f-a709-523a4b952c4d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.568327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bdc38-7922-4d7f-a709-523a4b952c4d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.569554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bdc38-7922-4d7f-a709-523a4b952c4d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:48.570515 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1207/2445] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:37:48.576419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c745-2bf6-4442-ad75-e7df5d5c6e35 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.576707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c745-2bf6-4442-ad75-e7df5d5c6e35 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.577009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171c745-2bf6-4442-ad75-e7df5d5c6e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:48.631467 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3171c745-2bf6-4442-ad75-e7df5d5c6e35 None None] MFA Rules not processed for user `13603c6cd9544f5391256ea600ad0709`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:48.633438 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3171c745-2bf6-4442-ad75-e7df5d5c6e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:48.703001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1239/2446] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:37:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:48.715958 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bdc38-7922-4d7f-a709-523a4b952c4d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:48.717699 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc2bdc38-7922-4d7f-a709-523a4b952c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:48.744776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bdc38-7922-4d7f-a709-523a4b952c4d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:48.748300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614f11d-53fc-4a25-8b84-8d5ec052c480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:48.748952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614f11d-53fc-4a25-8b84-8d5ec052c480 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.749233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614f11d-53fc-4a25-8b84-8d5ec052c480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.749502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614f11d-53fc-4a25-8b84-8d5ec052c480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:48.779257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614f11d-53fc-4a25-8b84-8d5ec052c480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:48.781270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614f11d-53fc-4a25-8b84-8d5ec052c480 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:48.794824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1208/2447] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:48.898705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2ef05-bb4c-4adc-aa71-9356feb2f1bf None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.899065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2ef05-bb4c-4adc-aa71-9356feb2f1bf None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.899359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2ef05-bb4c-4adc-aa71-9356feb2f1bf None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:48.901092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1240/2448] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:37:48.906678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70122b-6222-47da-b46b-c274beec9e74 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.907112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70122b-6222-47da-b46b-c274beec9e74 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.907330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70122b-6222-47da-b46b-c274beec9e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:48.932559 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7e70122b-6222-47da-b46b-c274beec9e74 None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:48.933803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7e70122b-6222-47da-b46b-c274beec9e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:48.953112 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1209/2449] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:37:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:48.965416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2ef05-bb4c-4adc-aa71-9356feb2f1bf None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:48.967564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-27d2ef05-bb4c-4adc-aa71-9356feb2f1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:48.985452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2ef05-bb4c-4adc-aa71-9356feb2f1bf None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:48.989689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d13ef-48ba-410e-82b2-7d2253bc9990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:48.990307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d13ef-48ba-410e-82b2-7d2253bc9990 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:48.990580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d13ef-48ba-410e-82b2-7d2253bc9990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:48.990912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d13ef-48ba-410e-82b2-7d2253bc9990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:49.009728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d13ef-48ba-410e-82b2-7d2253bc9990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:49.012371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d13ef-48ba-410e-82b2-7d2253bc9990 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:49.027583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1241/2450] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:37:52.559366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248d96a-3b37-499c-8d02-cc85ac68a4dc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:52.559901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248d96a-3b37-499c-8d02-cc85ac68a4dc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:52.559901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248d96a-3b37-499c-8d02-cc85ac68a4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:52.580061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c248d96a-3b37-499c-8d02-cc85ac68a4dc None None] MFA Rules not processed for user `6b1139bf460a49d2b0156912be60fa36`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:52.581293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c248d96a-3b37-499c-8d02-cc85ac68a4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:52.640232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1210/2451] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:37:53.313015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d13ef-48ba-410e-82b2-7d2253bc9990 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:53.398630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248d96a-3b37-499c-8d02-cc85ac68a4dc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:53.401022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c248d96a-3b37-499c-8d02-cc85ac68a4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:53.456975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d13ef-48ba-410e-82b2-7d2253bc9990 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:53.477123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:53.477995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:53.478472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:53.479199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] PATH_INFO: `/v3/users/8430fd71c41b46fd944013d15060d3a7` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:53.481504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8430fd71c41b46fd944013d15060d3a7)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:53.503417 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:53.507904 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:53.556123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248d96a-3b37-499c-8d02-cc85ac68a4dc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:53.560448 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:37:53.566610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecad6b-f0ff-433e-af8f-07f14fffb1c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:53.567832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecad6b-f0ff-433e-af8f-07f14fffb1c0 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:53.568337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecad6b-f0ff-433e-af8f-07f14fffb1c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:53.568467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecad6b-f0ff-433e-af8f-07f14fffb1c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:53.573405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-95ecad6b-f0ff-433e-af8f-07f14fffb1c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:53.593406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8430fd71c41b46fd944013d15060d3a7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:53.602535 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecad6b-f0ff-433e-af8f-07f14fffb1c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:53.604362 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecad6b-f0ff-433e-af8f-07f14fffb1c0 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:53.611890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8430fd71c41b46fd944013d15060d3a7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:53.612653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8430fd71c41b46fd944013d15060d3a7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:53.629616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8430fd71c41b46fd944013d15060d3a7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:53.630505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8430fd71c41b46fd944013d15060d3a7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:53.640316 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1211/2452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:37:53.651551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1242/2453] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:37:53 2022] DELETE /identity/v3/users/8430fd71c41b46fd944013d15060d3a7 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:37:53.654356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6dd2-7e90-4e1e-99d5-0946c72d14fe None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:53.654830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6dd2-7e90-4e1e-99d5-0946c72d14fe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:53.655214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6dd2-7e90-4e1e-99d5-0946c72d14fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:53.681232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0cab6dd2-7e90-4e1e-99d5-0946c72d14fe None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:37:53.682618 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0cab6dd2-7e90-4e1e-99d5-0946c72d14fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:53.777626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1212/2454] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:37:53.788816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:53.790766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:53.886830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b2ae-08ca-429f-abdb-659e0003ffa4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:53.890838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472c80e-01a7-420e-b2c6-8213a2e419be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:53.891458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472c80e-01a7-420e-b2c6-8213a2e419be service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:53.891736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472c80e-01a7-420e-b2c6-8213a2e419be service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:53.892007 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472c80e-01a7-420e-b2c6-8213a2e419be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:53.901674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472c80e-01a7-420e-b2c6-8213a2e419be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:53.903273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472c80e-01a7-420e-b2c6-8213a2e419be service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:53.913336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1243/2455] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:37:54.341351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab6dd2-7e90-4e1e-99d5-0946c72d14fe None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:37:54.343002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0cab6dd2-7e90-4e1e-99d5-0946c72d14fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:37:54.355608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab6dd2-7e90-4e1e-99d5-0946c72d14fe None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:37:54.361921 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:37:54.362571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:37:54.362878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:37:54.363186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] PATH_INFO: `/v3/projects/20cd24e91c574f60b16950db42a18b30` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:37:54.364394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20cd24e91c574f60b16950db42a18b30)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:37:54.366326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:37:54.423188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20cd24e91c574f60b16950db42a18b30', 'request_id': 'req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:54.430870 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20cd24e91c574f60b16950db42a18b30', 'request_id': 'req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:54.441979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20cd24e91c574f60b16950db42a18b30', 'request_id': 'req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:54.456004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] The token cache is being invalidate because project 20cd24e91c574f60b16950db42a18b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:37:54.456292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:54.457009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:37:54.459712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1213/2456] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:37:54 2022] DELETE /identity/v3/projects/20cd24e91c574f60b16950db42a18b30 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:38:02.943880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e493ead-7453-40a1-98b8-9b8bcf69286d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:02.944763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e493ead-7453-40a1-98b8-9b8bcf69286d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:02.945254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e493ead-7453-40a1-98b8-9b8bcf69286d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:02.968324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8e493ead-7453-40a1-98b8-9b8bcf69286d None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:38:02.975551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8e493ead-7453-40a1-98b8-9b8bcf69286d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:03.039212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1244/2457] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:38:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:38:03.052713 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:38:03.054216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:03.120585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:38:03.137164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e12965-bcde-4b8e-99c5-fd4ad246fb06 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:38:03.142702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74457e11-1fe0-407b-a4fc-03432a12727a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:38:03.143398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74457e11-1fe0-407b-a4fc-03432a12727a service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:03.143673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74457e11-1fe0-407b-a4fc-03432a12727a service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:03.143938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74457e11-1fe0-407b-a4fc-03432a12727a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:03.152815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74457e11-1fe0-407b-a4fc-03432a12727a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:38:03.154413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74457e11-1fe0-407b-a4fc-03432a12727a service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:38:03.163456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1214/2458] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:38:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:38:13.031742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=70,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:38:13.159590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:38:16.585536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665b4c2-3b25-4c45-869c-52ccde7b6c90 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:16.586054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665b4c2-3b25-4c45-869c-52ccde7b6c90 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:16.586054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665b4c2-3b25-4c45-869c-52ccde7b6c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:16.607709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7665b4c2-3b25-4c45-869c-52ccde7b6c90 None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:38:16.609891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7665b4c2-3b25-4c45-869c-52ccde7b6c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:16.617486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1245/2459] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:38:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:38:16.644367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74457e11-1fe0-407b-a4fc-03432a12727a service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:38:16.657009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74457e11-1fe0-407b-a4fc-03432a12727a service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:38:16.662247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e75b37-fb82-440d-acf3-260538b5bf41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:38:16.662893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e75b37-fb82-440d-acf3-260538b5bf41 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:16.663298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e75b37-fb82-440d-acf3-260538b5bf41 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:16.663573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e75b37-fb82-440d-acf3-260538b5bf41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:16.672475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e75b37-fb82-440d-acf3-260538b5bf41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:38:16.674058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e75b37-fb82-440d-acf3-260538b5bf41 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:38:16.682768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1215/2460] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:38:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:38:26.600912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:38:26.679192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:38:39.803946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7d5fc-9aa9-4c78-b8c2-a98093c617ce None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:39.804427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7d5fc-9aa9-4c78-b8c2-a98093c617ce None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:39.804518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7d5fc-9aa9-4c78-b8c2-a98093c617ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:39.824935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bcd7d5fc-9aa9-4c78-b8c2-a98093c617ce None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:38:39.826130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bcd7d5fc-9aa9-4c78-b8c2-a98093c617ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:39.834573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1246/2461] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:38:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:38:39.847227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e75b37-fb82-440d-acf3-260538b5bf41 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:38:39.858992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e75b37-fb82-440d-acf3-260538b5bf41 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:38:39.865098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f87f8-ebdf-4f78-95d9-0ca937754f5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:38:39.865702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f87f8-ebdf-4f78-95d9-0ca937754f5d service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:39.865976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f87f8-ebdf-4f78-95d9-0ca937754f5d service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:39.866241 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f87f8-ebdf-4f78-95d9-0ca937754f5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:39.874495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f87f8-ebdf-4f78-95d9-0ca937754f5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:38:39.876145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f87f8-ebdf-4f78-95d9-0ca937754f5d service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:38:39.884590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1216/2462] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:38:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:38:44.432700 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796b36e-92ef-4c86-ac7e-ad002a37687e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:44.433586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796b36e-92ef-4c86-ac7e-ad002a37687e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:44.433586 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796b36e-92ef-4c86-ac7e-ad002a37687e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:44.473119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b796b36e-92ef-4c86-ac7e-ad002a37687e None None] MFA Rules not processed for user `503ba8b1d3f64186b188bed35c4baefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:38:44.475783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b796b36e-92ef-4c86-ac7e-ad002a37687e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:44.560258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1247/2463] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:38:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:38:48.151912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7cef2-ef42-4af3-a9f2-ba0c8e10783a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:48.152871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7cef2-ef42-4af3-a9f2-ba0c8e10783a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:48.153352 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7cef2-ef42-4af3-a9f2-ba0c8e10783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:48.187846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9fb7cef2-ef42-4af3-a9f2-ba0c8e10783a None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:38:48.190994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9fb7cef2-ef42-4af3-a9f2-ba0c8e10783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:48.215031 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1217/2464] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 31 04:38:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:38:48.223274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796b36e-92ef-4c86-ac7e-ad002a37687e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:38:48.244678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796b36e-92ef-4c86-ac7e-ad002a37687e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:38:48.262975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f31465-756d-4e57-bb67-fdc7bc38c003 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:38:48.263879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f31465-756d-4e57-bb67-fdc7bc38c003 service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:48.265860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f31465-756d-4e57-bb67-fdc7bc38c003 service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:48.266959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f31465-756d-4e57-bb67-fdc7bc38c003 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:48.280128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f31465-756d-4e57-bb67-fdc7bc38c003 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:38:48.283185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f31465-756d-4e57-bb67-fdc7bc38c003 service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:38:48.292939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1248/2465] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:38:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:38:56.993222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7cef2-ef42-4af3-a9f2-ba0c8e10783a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:38:56.994830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9fb7cef2-ef42-4af3-a9f2-ba0c8e10783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:57.053032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7cef2-ef42-4af3-a9f2-ba0c8e10783a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:38:57.057202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:38:57.057807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:57.058074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:57.058341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:57.060224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:57.124929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:38:57.126324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:38:57.136282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1218/2466] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:38:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:38:57.321027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0e134-56a5-404c-a4f6-637389d9dc8e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:57.321494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0e134-56a5-404c-a4f6-637389d9dc8e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:57.321575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0e134-56a5-404c-a4f6-637389d9dc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:57.343621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2fd0e134-56a5-404c-a4f6-637389d9dc8e None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:38:57.344899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2fd0e134-56a5-404c-a4f6-637389d9dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:38:57.352653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1249/2467] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 31 04:38:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:38:57.366022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:38:57.380126 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf2a14-7055-4dd7-a631-1733b58e5c52 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:38:57.384986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7c5e4-2b35-414b-9fcf-122585311bed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:38:57.385601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c5e4-2b35-414b-9fcf-122585311bed service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:38:57.385938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c5e4-2b35-414b-9fcf-122585311bed service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:38:57.386230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7c5e4-2b35-414b-9fcf-122585311bed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:38:57.396635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7c5e4-2b35-414b-9fcf-122585311bed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:38:57.398107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7c5e4-2b35-414b-9fcf-122585311bed service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:38:57.422700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1219/2468] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:39:00.457566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0e134-56a5-404c-a4f6-637389d9dc8e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:00.459888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2fd0e134-56a5-404c-a4f6-637389d9dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:00.541718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0e134-56a5-404c-a4f6-637389d9dc8e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:00.546654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:00.547404 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:00.547758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:00.548410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] PATH_INFO: `/v3/users/fffc906c79ba418cbc52e5db5ca991ca` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:00.549940 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fffc906c79ba418cbc52e5db5ca991ca)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:00.551937 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:00.607064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fffc906c79ba418cbc52e5db5ca991ca'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:00.617598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fffc906c79ba418cbc52e5db5ca991ca'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:00.618459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fffc906c79ba418cbc52e5db5ca991ca'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:00.634102 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fffc906c79ba418cbc52e5db5ca991ca'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:00.635003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e5da5065-5e50-41c6-8fdf-9cfc3736631d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fffc906c79ba418cbc52e5db5ca991ca'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:00.646143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:39:00.649227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1250/2469] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:39:00 2022] DELETE /identity/v3/users/fffc906c79ba418cbc52e5db5ca991ca => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:39:00.899564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7c5e4-2b35-414b-9fcf-122585311bed service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:00.913522 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7c5e4-2b35-414b-9fcf-122585311bed service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:00.987501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:00.988175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:00.988456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:00.988726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] PATH_INFO: `/v3/projects/b35a44f3c0ba4230bac2bcb9d4e8ec08` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:00.989842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35a44f3c0ba4230bac2bcb9d4e8ec08)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:00.991787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:01.013615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:39:01.041328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35a44f3c0ba4230bac2bcb9d4e8ec08', 'request_id': 'req-7e110620-5cb8-4965-83f6-b13815cae796'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:01.048162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35a44f3c0ba4230bac2bcb9d4e8ec08', 'request_id': 'req-7e110620-5cb8-4965-83f6-b13815cae796'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:01.057945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35a44f3c0ba4230bac2bcb9d4e8ec08', 'request_id': 'req-7e110620-5cb8-4965-83f6-b13815cae796'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:01.073188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] The token cache is being invalidate because project b35a44f3c0ba4230bac2bcb9d4e8ec08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:39:01.073311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:01.074109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7e110620-5cb8-4965-83f6-b13815cae796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:01.075919 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1220/2470] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:39:00 2022] DELETE /identity/v3/projects/b35a44f3c0ba4230bac2bcb9d4e8ec08 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:39:10.660402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:39:11.069769 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:39:23.897670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58786-024b-4094-90f0-924494e904c1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:23.898239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58786-024b-4094-90f0-924494e904c1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:23.898239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58786-024b-4094-90f0-924494e904c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:23.918994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-07f58786-024b-4094-90f0-924494e904c1 None None] MFA Rules not processed for user `863e457bd6fd4bff82f552e63ca1034c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:39:23.920208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-07f58786-024b-4094-90f0-924494e904c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:23.981576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1251/2471] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:39:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:39:33.974074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=25 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:39:50.370146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028879d-fd85-46b9-82fa-265edcb6684b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:50.370744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028879d-fd85-46b9-82fa-265edcb6684b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:50.370744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028879d-fd85-46b9-82fa-265edcb6684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:50.390916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1028879d-fd85-46b9-82fa-265edcb6684b None None] MFA Rules not processed for user `13ac62bdb9bc4c7f94dad08f810559e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:39:50.392128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1028879d-fd85-46b9-82fa-265edcb6684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:50.455558 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1221/2472] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:39:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:39:50.494678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f58786-024b-4094-90f0-924494e904c1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:50.496192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-07f58786-024b-4094-90f0-924494e904c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:50.557183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f58786-024b-4094-90f0-924494e904c1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:50.561706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:50.562292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:50.562566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:50.562874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:50.564657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:50.642908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:50.644325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:50.655516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1252/2473] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:39:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:39:50.937049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028879d-fd85-46b9-82fa-265edcb6684b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:50.938453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1028879d-fd85-46b9-82fa-265edcb6684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:50.995339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028879d-fd85-46b9-82fa-265edcb6684b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:50.999152 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:50.999619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:50.999900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:51.000176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:51.001979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:51.011776 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:51.013155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:51.021764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1222/2474] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:39:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:39:52.338038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:52.340129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:52.355059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eee932-6144-4aba-a57e-e1f6bb13db4f service glance] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:52.359460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:52.360969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:52.362540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:52.363261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:52.363526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:52.363789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:52.365587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:52.376807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:52.378387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:52.387753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1253/2475] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:39:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:39:52.421856 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:52.423589 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:52.431710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abbc26-1a5e-4cf2-9783-642577f5ba20 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:52.435319 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d5606-e615-4b99-8e93-9b25e4171758 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:52.437003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:52.437622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:52.437929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:52.438207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:52.439855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:52.440080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:52.440782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:52.441061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:52.441444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:52.443157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:52.451736 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:52.453351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:52.457621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:52.459116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:52.463651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1223/2476] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:39:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:39:52.470830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1254/2477] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:39:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:39:55.908391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:55.909925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:55.968842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:39:55.991287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d917-3ddd-4511-b6be-8004fa8b170b service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:56.005672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:56.006684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:56.007243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:56.007519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:56.009350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:56.090353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:56.091825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:56.101551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1224/2478] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:39:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:39:56.356495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:56.358857 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:39:56.432514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec509d-5005-4603-ac9e-63a00193956d service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:56.437790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:56.438599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:56.438908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:56.439237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] PATH_INFO: `/v3/users/503ba8b1d3f64186b188bed35c4baefe` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:56.441180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=503ba8b1d3f64186b188bed35c4baefe)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:56.444301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:56.462470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:39:56.520105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503ba8b1d3f64186b188bed35c4baefe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.532188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503ba8b1d3f64186b188bed35c4baefe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.532912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503ba8b1d3f64186b188bed35c4baefe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.547162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503ba8b1d3f64186b188bed35c4baefe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.548142 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503ba8b1d3f64186b188bed35c4baefe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.562410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1255/2479] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:39:56 2022] DELETE /identity/v3/users/503ba8b1d3f64186b188bed35c4baefe => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:39:56.751081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:39:56.762829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd7ebc-d293-468a-9523-7d0fdd932c6d service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:39:56.821368 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:39:56.822003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:39:56.822278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:39:56.822551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] PATH_INFO: `/v3/projects/52fbe77f79a5449b8fb2bfef82360b94` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:39:56.823710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52fbe77f79a5449b8fb2bfef82360b94)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:39:56.825651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:39:56.874975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52fbe77f79a5449b8fb2bfef82360b94', 'request_id': 'req-2e6f44e0-7d2a-41c9-ba50-8591369b1733'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.881279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52fbe77f79a5449b8fb2bfef82360b94', 'request_id': 'req-2e6f44e0-7d2a-41c9-ba50-8591369b1733'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.890743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52fbe77f79a5449b8fb2bfef82360b94', 'request_id': 'req-2e6f44e0-7d2a-41c9-ba50-8591369b1733'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.901923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] The token cache is being invalidate because project 52fbe77f79a5449b8fb2bfef82360b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:39:56.902205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.902899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2e6f44e0-7d2a-41c9-ba50-8591369b1733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:39:56.904490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1225/2480] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:39:56 2022] DELETE /identity/v3/projects/52fbe77f79a5449b8fb2bfef82360b94 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:40:03.971751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:03.973264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:04.036969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859f85d-c530-4d6c-bfe7-c79fc2bf7a4c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:04.094507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:04.095161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:04.095436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:04.095707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:04.097516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:04.108024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:04.109778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:04.117851 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1256/2481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:04.573941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8266f5-71ed-41db-935c-09d54b3da7c6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:04.574463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8266f5-71ed-41db-935c-09d54b3da7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:04.574463 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8266f5-71ed-41db-935c-09d54b3da7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:04.595243 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cb8266f5-71ed-41db-935c-09d54b3da7c6 None None] MFA Rules not processed for user `718418f954f94dad93eeb725e345c850`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:04.596460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cb8266f5-71ed-41db-935c-09d54b3da7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:04.617391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:04.658276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1226/2482] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:04.986108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:04.987645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:04.998103 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0422c5-8447-4dde-8d09-6f415a34925d service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:05.001931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:05.002529 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:05.002843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:05.003094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:05.004748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:05.013809 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:05.015312 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:05.023249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1257/2483] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:08.432172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8266f5-71ed-41db-935c-09d54b3da7c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:08.433836 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cb8266f5-71ed-41db-935c-09d54b3da7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:08.504873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8266f5-71ed-41db-935c-09d54b3da7c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:08.508690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:08.509443 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:08.509721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:08.509991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:08.511823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:08.560914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:08.562363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:08.585977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:08.588309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:08.615753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:08.619086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1227/2484] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:08.692197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cea97-20f1-4c4c-871e-1d156347fd8b service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:08.702786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:08.705294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:08.705741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:08.706183 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:08.711805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:08.727946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:08.730230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:08.745713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1258/2485] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:18.596584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=63 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:18.740116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=16 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:31.242130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:31.252567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f7460-a5e3-4d53-b415-1dce9a9c917b service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:31.258131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:31.259065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:31.259442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:31.259767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:31.263569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:31.344267 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:31.345708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:31.356735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1228/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:34.649414 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:34.651073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:34.668612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981affc-6b97-415b-8a0f-b408ffd4eb0a service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:34.673612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:34.674253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:34.674530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:34.674891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] PATH_INFO: `/v3/users/863e457bd6fd4bff82f552e63ca1034c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:34.676013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863e457bd6fd4bff82f552e63ca1034c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:34.678016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:34.725952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863e457bd6fd4bff82f552e63ca1034c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:34.734582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863e457bd6fd4bff82f552e63ca1034c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:34.735278 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863e457bd6fd4bff82f552e63ca1034c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:34.747537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863e457bd6fd4bff82f552e63ca1034c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:34.748191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bc559f2f-55aa-442c-9d52-f5fbaa02cb02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863e457bd6fd4bff82f552e63ca1034c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:34.760189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1259/2487] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:40:34 2022] DELETE /identity/v3/users/863e457bd6fd4bff82f552e63ca1034c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:40:34.941589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:34.955350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127b661-8abc-4519-bd43-0703cdcf1dd3 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:35.016372 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:35.017026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.017297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.017557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] PATH_INFO: `/v3/projects/c02e60929267489cb181831f1ac578e4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.018643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02e60929267489cb181831f1ac578e4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:35.020605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:35.070482 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02e60929267489cb181831f1ac578e4', 'request_id': 'req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:35.076781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02e60929267489cb181831f1ac578e4', 'request_id': 'req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:35.086110 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02e60929267489cb181831f1ac578e4', 'request_id': 'req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:35.097226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] The token cache is being invalidate because project c02e60929267489cb181831f1ac578e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:40:35.097508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:35.098153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bebe7506-369e-4f5d-b6f5-a0144a87f7f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:35.099711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1229/2488] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:40:34 2022] DELETE /identity/v3/projects/c02e60929267489cb181831f1ac578e4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:40:35.104805 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208414d-16a8-4ced-98e2-641ec575f75f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.105112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208414d-16a8-4ced-98e2-641ec575f75f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.105923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208414d-16a8-4ced-98e2-641ec575f75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.126912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8208414d-16a8-4ced-98e2-641ec575f75f None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:35.128151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8208414d-16a8-4ced-98e2-641ec575f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:35.186138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1260/2489] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:35.194053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da405c04-a8ad-4dfe-b7d5-85a4fce7f485 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.194333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da405c04-a8ad-4dfe-b7d5-85a4fce7f485 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.194605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da405c04-a8ad-4dfe-b7d5-85a4fce7f485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.215563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-da405c04-a8ad-4dfe-b7d5-85a4fce7f485 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:35.216785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-da405c04-a8ad-4dfe-b7d5-85a4fce7f485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:35.226317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1230/2490] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:35.231322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208414d-16a8-4ced-98e2-641ec575f75f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:35.242220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208414d-16a8-4ced-98e2-641ec575f75f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:35.248001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:35.248603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.248882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.249150 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.249801 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:40:35.250110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:35.251747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:35.263765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1261/2491] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:40:35.269088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da405c04-a8ad-4dfe-b7d5-85a4fce7f485 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:35.284500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da405c04-a8ad-4dfe-b7d5-85a4fce7f485 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:35.293639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac7984-46f8-4522-97ff-689854c098a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:35.294251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac7984-46f8-4522-97ff-689854c098a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.294585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac7984-46f8-4522-97ff-689854c098a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.294886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac7984-46f8-4522-97ff-689854c098a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.295508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac7984-46f8-4522-97ff-689854c098a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:35.297399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac7984-46f8-4522-97ff-689854c098a8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:35.314178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:35.331991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1231/2492] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:40:35 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:40:35.336846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:35.347696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257ca1c-af7f-42ef-8165-2ca1d9c128b6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:35.417444 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9a032-0ff4-41f7-b78b-a21bf2c10919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:35.418316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a032-0ff4-41f7-b78b-a21bf2c10919 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.418594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a032-0ff4-41f7-b78b-a21bf2c10919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.418897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a032-0ff4-41f7-b78b-a21bf2c10919 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.419518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9a032-0ff4-41f7-b78b-a21bf2c10919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:35.421416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9a032-0ff4-41f7-b78b-a21bf2c10919 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:35.433277 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:35.457991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1262/2493] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:40:35 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:40:35.463948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac7984-46f8-4522-97ff-689854c098a8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:35.475626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac7984-46f8-4522-97ff-689854c098a8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:35.482319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf0caa-cb96-4e47-99c3-4704d6953bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:35.482959 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf0caa-cb96-4e47-99c3-4704d6953bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.483215 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf0caa-cb96-4e47-99c3-4704d6953bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.483595 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf0caa-cb96-4e47-99c3-4704d6953bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.484156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf0caa-cb96-4e47-99c3-4704d6953bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:35.485735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf0caa-cb96-4e47-99c3-4704d6953bc1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:35.496632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1232/2494] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:40:35.500981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9a032-0ff4-41f7-b78b-a21bf2c10919 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:35.516408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9a032-0ff4-41f7-b78b-a21bf2c10919 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:35.524225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127ad8f-1d4d-4a2d-bf7b-75021ae6e3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:35.524842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127ad8f-1d4d-4a2d-bf7b-75021ae6e3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.525119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127ad8f-1d4d-4a2d-bf7b-75021ae6e3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.525366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127ad8f-1d4d-4a2d-bf7b-75021ae6e3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.526024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127ad8f-1d4d-4a2d-bf7b-75021ae6e3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:35.527771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127ad8f-1d4d-4a2d-bf7b-75021ae6e3ad admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:35.538958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1263/2495] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:40:35.544307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf0caa-cb96-4e47-99c3-4704d6953bc1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:35.554823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf0caa-cb96-4e47-99c3-4704d6953bc1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:35.561016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:35.561709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.561988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.562254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] PATH_INFO: `/v3/projects/ae904e4b1df846e497cdf0edc46edaf1/users/9f4dcda6ad1d401a9aed637906563d72/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.578693 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae904e4b1df846e497cdf0edc46edaf1, user_id=9f4dcda6ad1d401a9aed637906563d72, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:35.581608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:35.596117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1233/2496] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:40:35 2022] PUT /identity/v3/projects/ae904e4b1df846e497cdf0edc46edaf1/users/9f4dcda6ad1d401a9aed637906563d72/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:40:35.601554 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47324-f5c4-4088-b51a-7809918acf26 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.601870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47324-f5c4-4088-b51a-7809918acf26 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.602144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b47324-f5c4-4088-b51a-7809918acf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.622754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-15b47324-f5c4-4088-b51a-7809918acf26 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:35.623951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-15b47324-f5c4-4088-b51a-7809918acf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:35.686773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1264/2497] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:35.700068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:35.701785 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:35.768832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fcb5c-87b7-4c0d-a748-fdd4d92cd694 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:35.772894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e080e6-924c-4064-af8a-5d11f3fdfcc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:35.773527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e080e6-924c-4064-af8a-5d11f3fdfcc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:35.773846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e080e6-924c-4064-af8a-5d11f3fdfcc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:35.774102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e080e6-924c-4064-af8a-5d11f3fdfcc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:35.784429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e080e6-924c-4064-af8a-5d11f3fdfcc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:35.785878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e080e6-924c-4064-af8a-5d11f3fdfcc4 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:35.795698 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1234/2498] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:36.294725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b47324-f5c4-4088-b51a-7809918acf26 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:36.296508 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-15b47324-f5c4-4088-b51a-7809918acf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:36.390285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b47324-f5c4-4088-b51a-7809918acf26 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:36.396915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:36.397745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:36.398075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:36.398355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:36.400496 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:36.419630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:36.421677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:36.435340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1265/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:40:41.130788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418ed4-4024-4e35-9e14-e444425dcd7e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:41.132288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418ed4-4024-4e35-9e14-e444425dcd7e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:41.132722 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418ed4-4024-4e35-9e14-e444425dcd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:41.165092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ad418ed4-4024-4e35-9e14-e444425dcd7e None None] MFA Rules not processed for user `9f4dcda6ad1d401a9aed637906563d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:41.166336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad418ed4-4024-4e35-9e14-e444425dcd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:41.242426 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1235/2500] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:40:41.257274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:41.272621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b9eb7-7e68-4cea-a078-8740865e51c6 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:41.281410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eb497-1f64-4fb3-b8e5-741f896e4259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:41.282056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eb497-1f64-4fb3-b8e5-741f896e4259 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:41.282338 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eb497-1f64-4fb3-b8e5-741f896e4259 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:41.282642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eb497-1f64-4fb3-b8e5-741f896e4259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:41.292235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eb497-1f64-4fb3-b8e5-741f896e4259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:41.294169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eb497-1f64-4fb3-b8e5-741f896e4259 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:41.305055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1266/2501] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:42.837376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1703aa-6b97-40e0-bb36-0dad65319c4a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:42.837868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1703aa-6b97-40e0-bb36-0dad65319c4a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:42.837964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1703aa-6b97-40e0-bb36-0dad65319c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:42.861428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ec1703aa-6b97-40e0-bb36-0dad65319c4a None None] MFA Rules not processed for user `9f4dcda6ad1d401a9aed637906563d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:42.863312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec1703aa-6b97-40e0-bb36-0dad65319c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:42.872875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1236/2502] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:44.932318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a157fad-ded5-4747-bd4a-c2938a7e0945 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:44.932818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a157fad-ded5-4747-bd4a-c2938a7e0945 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:44.933174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a157fad-ded5-4747-bd4a-c2938a7e0945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:44.953660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8a157fad-ded5-4747-bd4a-c2938a7e0945 None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:44.955002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8a157fad-ded5-4747-bd4a-c2938a7e0945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:45.020766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:45.037030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1267/2503] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 31 04:40:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:45.104078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1703aa-6b97-40e0-bb36-0dad65319c4a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:45.106210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec1703aa-6b97-40e0-bb36-0dad65319c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:45.173149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:45.198056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1703aa-6b97-40e0-bb36-0dad65319c4a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:45.205440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6da5a0-c908-48cd-bb9d-d826134e2d90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:45.206449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6da5a0-c908-48cd-bb9d-d826134e2d90 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:45.206995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6da5a0-c908-48cd-bb9d-d826134e2d90 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:45.207490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6da5a0-c908-48cd-bb9d-d826134e2d90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:45.219092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6da5a0-c908-48cd-bb9d-d826134e2d90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:45.221418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6da5a0-c908-48cd-bb9d-d826134e2d90 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:45.232265 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1237/2504] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:40:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:40:48.971040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a157fad-ded5-4747-bd4a-c2938a7e0945 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:48.972651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8a157fad-ded5-4747-bd4a-c2938a7e0945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:49.051369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a157fad-ded5-4747-bd4a-c2938a7e0945 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:49.056790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:49.057355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:49.057750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:49.058027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:49.060153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:49.079546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:49.081339 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:49.092060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1268/2505] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:52.178057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6da5a0-c908-48cd-bb9d-d826134e2d90 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:52.196835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6da5a0-c908-48cd-bb9d-d826134e2d90 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:52.208416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:52.209436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:52.209757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:52.210131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:52.212822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:52.228439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:52.230624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:52.241469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:52.245606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:52.262378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dc5bc-0966-4c78-bf61-976d49d8c8ae service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:52.273172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1238/2506] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:52.280389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:52.282887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:52.283332 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:52.284095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:52.287113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:52.303917 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:52.306548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:52.345064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:52.346639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:52.363532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1269/2507] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:40:52.370747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b39ed8-105d-4ef2-ba81-794d2011ea96 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:52.384382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:52.385759 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:52.396224 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:52.396583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] PATH_INFO: `/v3/users/6b1139bf460a49d2b0156912be60fa36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:52.400948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b1139bf460a49d2b0156912be60fa36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:52.408234 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:52.497023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b1139bf460a49d2b0156912be60fa36'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.505320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b1139bf460a49d2b0156912be60fa36'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.506009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b1139bf460a49d2b0156912be60fa36'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.519097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b1139bf460a49d2b0156912be60fa36'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.519812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b1139bf460a49d2b0156912be60fa36'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.535850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1239/2508] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:40:52 2022] DELETE /identity/v3/users/6b1139bf460a49d2b0156912be60fa36 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:40:52.717619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:52.728607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7bc91-68aa-4863-aef8-6784e7f0553c service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:52.791445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:52.792080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:52.792358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:52.792625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] PATH_INFO: `/v3/projects/6714373199f5419a9a70267187067934` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:52.793907 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6714373199f5419a9a70267187067934)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:52.795891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:52.845142 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6714373199f5419a9a70267187067934', 'request_id': 'req-aea6d001-0a1d-4cc6-b785-491d60cd057d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.854486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6714373199f5419a9a70267187067934', 'request_id': 'req-aea6d001-0a1d-4cc6-b785-491d60cd057d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.862198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6714373199f5419a9a70267187067934', 'request_id': 'req-aea6d001-0a1d-4cc6-b785-491d60cd057d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.875772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] The token cache is being invalidate because project 6714373199f5419a9a70267187067934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:40:52.876090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.876771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:52.878294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1270/2509] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:40:52 2022] DELETE /identity/v3/projects/6714373199f5419a9a70267187067934 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:40:52.882683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:52.884753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:52.903858 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6363bc7-0014-440e-836b-2d22be67ce27 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:52.964210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:52.964843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:52.965122 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:52.965495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] PATH_INFO: `/v3/users/5ddb77df65c345409f592ad86032f84c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:52.966667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddb77df65c345409f592ad86032f84c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:52.968628 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:53.010131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:53.019317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddb77df65c345409f592ad86032f84c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.028371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddb77df65c345409f592ad86032f84c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.028978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddb77df65c345409f592ad86032f84c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.040620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddb77df65c345409f592ad86032f84c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.041264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f3ab3b7f-259b-4b70-81ea-952ec876a523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddb77df65c345409f592ad86032f84c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.053297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1240/2510] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:40:52 2022] DELETE /identity/v3/users/5ddb77df65c345409f592ad86032f84c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:40:53.236819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:53.238490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:53.246399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:53.253933 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6d001-0a1d-4cc6-b785-491d60cd057d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:53.319898 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:53.320585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.320857 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.321129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] PATH_INFO: `/v3/projects/f992ae8c1f3b47dd88bd3f2b20216125` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.322368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f992ae8c1f3b47dd88bd3f2b20216125)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:53.324359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:53.374151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f992ae8c1f3b47dd88bd3f2b20216125', 'request_id': 'req-c018fb49-5436-4f49-800d-9534d437af20'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.381830 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f992ae8c1f3b47dd88bd3f2b20216125', 'request_id': 'req-c018fb49-5436-4f49-800d-9534d437af20'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.388275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f992ae8c1f3b47dd88bd3f2b20216125', 'request_id': 'req-c018fb49-5436-4f49-800d-9534d437af20'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.398757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] The token cache is being invalidate because project f992ae8c1f3b47dd88bd3f2b20216125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:40:53.399051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.399679 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c018fb49-5436-4f49-800d-9534d437af20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:40:53.401175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1271/2511] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:40:53 2022] DELETE /identity/v3/projects/f992ae8c1f3b47dd88bd3f2b20216125 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:40:53.406288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee136-eebe-4cd0-a418-655e017d2848 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.406805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee136-eebe-4cd0-a418-655e017d2848 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.406909 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee136-eebe-4cd0-a418-655e017d2848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.427428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-276ee136-eebe-4cd0-a418-655e017d2848 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:53.428615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-276ee136-eebe-4cd0-a418-655e017d2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:53.506936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1241/2512] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:53.516030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6beeff-1c80-4f52-a10c-b55a7fc7a3e2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.516195 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6beeff-1c80-4f52-a10c-b55a7fc7a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.516497 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6beeff-1c80-4f52-a10c-b55a7fc7a3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.537533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5d6beeff-1c80-4f52-a10c-b55a7fc7a3e2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:53.538767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5d6beeff-1c80-4f52-a10c-b55a7fc7a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:53.547578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1272/2513] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:53.553382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ee136-eebe-4cd0-a418-655e017d2848 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:53.564588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ee136-eebe-4cd0-a418-655e017d2848 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:53.569346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:53.569976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.570254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.570525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.571229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:40:53.571718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:53.573893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:53.587757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1242/2514] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:40:53.592807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6beeff-1c80-4f52-a10c-b55a7fc7a3e2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:53.603305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6beeff-1c80-4f52-a10c-b55a7fc7a3e2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:53.611673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988cb83-e207-4db1-9d95-b9a5c78f58e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:53.612379 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988cb83-e207-4db1-9d95-b9a5c78f58e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.612687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988cb83-e207-4db1-9d95-b9a5c78f58e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.612948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988cb83-e207-4db1-9d95-b9a5c78f58e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.615425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988cb83-e207-4db1-9d95-b9a5c78f58e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:53.617563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988cb83-e207-4db1-9d95-b9a5c78f58e8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:53.653977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1273/2515] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:40:53 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:40:53.657750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:53.668927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d164ce9-68bb-4fca-adc0-6657138c1591 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:53.730005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62cf9f-bcba-4a71-837a-e6a1ac10f9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:53.730621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62cf9f-bcba-4a71-837a-e6a1ac10f9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.731663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62cf9f-bcba-4a71-837a-e6a1ac10f9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.732192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62cf9f-bcba-4a71-837a-e6a1ac10f9d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.732542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62cf9f-bcba-4a71-837a-e6a1ac10f9d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:53.734458 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62cf9f-bcba-4a71-837a-e6a1ac10f9d1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:53.757635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1243/2516] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:40:53 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:40:53.761858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988cb83-e207-4db1-9d95-b9a5c78f58e8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:53.773398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988cb83-e207-4db1-9d95-b9a5c78f58e8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:53.778503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe74ea0-957c-46a1-afe2-f4a3c95bd9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:53.779159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe74ea0-957c-46a1-afe2-f4a3c95bd9dd admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.779445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe74ea0-957c-46a1-afe2-f4a3c95bd9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.779722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe74ea0-957c-46a1-afe2-f4a3c95bd9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.780377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe74ea0-957c-46a1-afe2-f4a3c95bd9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:53.782058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe74ea0-957c-46a1-afe2-f4a3c95bd9dd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:53.792124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1274/2517] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:40:53.797993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62cf9f-bcba-4a71-837a-e6a1ac10f9d1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:53.808453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62cf9f-bcba-4a71-837a-e6a1ac10f9d1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:53.815325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770d7a5-9b4d-4c29-b23b-62c39e4d30ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:53.815964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d7a5-9b4d-4c29-b23b-62c39e4d30ad admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.816276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d7a5-9b4d-4c29-b23b-62c39e4d30ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.816552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d7a5-9b4d-4c29-b23b-62c39e4d30ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.817277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770d7a5-9b4d-4c29-b23b-62c39e4d30ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:53.818945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770d7a5-9b4d-4c29-b23b-62c39e4d30ad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:53.833629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1244/2518] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:40:53.837792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe74ea0-957c-46a1-afe2-f4a3c95bd9dd admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:53.850075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe74ea0-957c-46a1-afe2-f4a3c95bd9dd admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:53.855205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:53.856512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.856912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.857429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] PATH_INFO: `/v3/projects/312b9335479c4d72b78a20bb2f1b7296/users/c11d44667e8e4db5bccccaee16d89a51/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.874340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312b9335479c4d72b78a20bb2f1b7296, user_id=c11d44667e8e4db5bccccaee16d89a51, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:53.877088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:53.891919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1275/2519] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:40:53 2022] PUT /identity/v3/projects/312b9335479c4d72b78a20bb2f1b7296/users/c11d44667e8e4db5bccccaee16d89a51/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:40:53.897098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bf6dc-5561-4923-bad3-1a0e573df8ae None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:53.897382 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bf6dc-5561-4923-bad3-1a0e573df8ae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:53.897645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bf6dc-5561-4923-bad3-1a0e573df8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:53.917162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1a6bf6dc-5561-4923-bad3-1a0e573df8ae None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:53.918422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1a6bf6dc-5561-4923-bad3-1a0e573df8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:53.933341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:53.994186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1245/2520] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:54.005794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:54.008137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:54.050816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:40:54.087569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb2d86-652a-4e63-a2a8-85d3ea2ddee5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:54.092034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae817761-8199-4f3d-929d-69e286b22dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:54.092941 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae817761-8199-4f3d-929d-69e286b22dda service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:54.093222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae817761-8199-4f3d-929d-69e286b22dda service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:54.093493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae817761-8199-4f3d-929d-69e286b22dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:54.103872 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae817761-8199-4f3d-929d-69e286b22dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:54.105303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae817761-8199-4f3d-929d-69e286b22dda service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:54.115479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1276/2521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:40:58.782655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e417-30b2-4c69-9c6e-591432667fc1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:58.783130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e417-30b2-4c69-9c6e-591432667fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:58.783225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e417-30b2-4c69-9c6e-591432667fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:58.814116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3676e417-30b2-4c69-9c6e-591432667fc1 None None] MFA Rules not processed for user `c11d44667e8e4db5bccccaee16d89a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:40:58.815530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3676e417-30b2-4c69-9c6e-591432667fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:40:58.905735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1246/2522] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:40:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:40:58.910374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae817761-8199-4f3d-929d-69e286b22dda service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:58.920432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae817761-8199-4f3d-929d-69e286b22dda service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:58.924898 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83cbcb-353d-4d4e-aea4-8b48e2d6fadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:58.925488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83cbcb-353d-4d4e-aea4-8b48e2d6fadc admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:58.925765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83cbcb-353d-4d4e-aea4-8b48e2d6fadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:58.926033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83cbcb-353d-4d4e-aea4-8b48e2d6fadc admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:58.929037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83cbcb-353d-4d4e-aea4-8b48e2d6fadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:58.929037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83cbcb-353d-4d4e-aea4-8b48e2d6fadc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:58.960787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1277/2523] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:40:58 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:40:58.964902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e417-30b2-4c69-9c6e-591432667fc1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:58.976256 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e417-30b2-4c69-9c6e-591432667fc1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:59.037824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab10fd5-c28c-4370-bc6b-9145e77d9f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:59.038454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab10fd5-c28c-4370-bc6b-9145e77d9f9c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:59.038757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab10fd5-c28c-4370-bc6b-9145e77d9f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:59.039036 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab10fd5-c28c-4370-bc6b-9145e77d9f9c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:59.039681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab10fd5-c28c-4370-bc6b-9145e77d9f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:59.041620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab10fd5-c28c-4370-bc6b-9145e77d9f9c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:59.071825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1247/2524] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:40:58 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:40:59.075111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83cbcb-353d-4d4e-aea4-8b48e2d6fadc admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:59.086111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83cbcb-353d-4d4e-aea4-8b48e2d6fadc admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:59.091353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca17478-cb9d-44c1-83d4-c155ab767a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:59.091971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17478-cb9d-44c1-83d4-c155ab767a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:59.092252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17478-cb9d-44c1-83d4-c155ab767a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:59.092638 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17478-cb9d-44c1-83d4-c155ab767a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:59.093157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca17478-cb9d-44c1-83d4-c155ab767a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:59.094860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca17478-cb9d-44c1-83d4-c155ab767a52 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:59.106705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1278/2525] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:40:59.111350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab10fd5-c28c-4370-bc6b-9145e77d9f9c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:59.124219 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab10fd5-c28c-4370-bc6b-9145e77d9f9c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:59.129136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3ae13-43fc-433a-885d-9016b1eb5c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:59.129820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3ae13-43fc-433a-885d-9016b1eb5c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:59.130097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3ae13-43fc-433a-885d-9016b1eb5c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:59.130615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3ae13-43fc-433a-885d-9016b1eb5c79 admin admin] PATH_INFO: `/v3/projects/0c6b37ba935a4a1f9a7795d0f873a2da/users/7595fe53d9bb416c8054fcedf96fc32f/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:59.147584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3ae13-43fc-433a-885d-9016b1eb5c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6b37ba935a4a1f9a7795d0f873a2da, user_id=7595fe53d9bb416c8054fcedf96fc32f, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:59.150268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3ae13-43fc-433a-885d-9016b1eb5c79 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:59.166201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1248/2526] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:40:59 2022] PUT /identity/v3/projects/0c6b37ba935a4a1f9a7795d0f873a2da/users/7595fe53d9bb416c8054fcedf96fc32f/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:40:59.170836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca17478-cb9d-44c1-83d4-c155ab767a52 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:59.181006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca17478-cb9d-44c1-83d4-c155ab767a52 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:59.240096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cba6f7-f39f-47e1-b99a-b40be0f6a931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:59.240758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cba6f7-f39f-47e1-b99a-b40be0f6a931 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:59.241036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cba6f7-f39f-47e1-b99a-b40be0f6a931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:59.241308 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cba6f7-f39f-47e1-b99a-b40be0f6a931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:59.241961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cba6f7-f39f-47e1-b99a-b40be0f6a931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:59.243806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cba6f7-f39f-47e1-b99a-b40be0f6a931 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:59.257728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1279/2527] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:40:59.259021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3ae13-43fc-433a-885d-9016b1eb5c79 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:40:59.270380 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3ae13-43fc-433a-885d-9016b1eb5c79 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:40:59.275253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:40:59.276004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:40:59.276347 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:40:59.276715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] PATH_INFO: `/v3/projects/0c6b37ba935a4a1f9a7795d0f873a2da/users/7595fe53d9bb416c8054fcedf96fc32f/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:40:59.279373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6b37ba935a4a1f9a7795d0f873a2da, user_id=7595fe53d9bb416c8054fcedf96fc32f, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:40:59.282062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:40:59.293039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1249/2528] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:40:59 2022] PUT /identity/v3/projects/0c6b37ba935a4a1f9a7795d0f873a2da/users/7595fe53d9bb416c8054fcedf96fc32f/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:03.910070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef263-b51f-4c7a-aec7-93b96e7c6251 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:03.910646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef263-b51f-4c7a-aec7-93b96e7c6251 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:03.910646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef263-b51f-4c7a-aec7-93b96e7c6251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:03.942490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1d3ef263-b51f-4c7a-aec7-93b96e7c6251 None None] MFA Rules not processed for user `7595fe53d9bb416c8054fcedf96fc32f`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:03.943650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1d3ef263-b51f-4c7a-aec7-93b96e7c6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:04.070691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1280/2529] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:04.094804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:04.096590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:04.102240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:04.191710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7016f4-3b36-44c6-a18b-84a6736a0067 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:04.203802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:04.204518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:04.204893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:04.205199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:04.271789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:04.273415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:04.286943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1250/2530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:07.818841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550bd3d-7d90-4632-a0e0-4dfabe363eca None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:07.819336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550bd3d-7d90-4632-a0e0-4dfabe363eca None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:07.819433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550bd3d-7d90-4632-a0e0-4dfabe363eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:07.842519 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c550bd3d-7d90-4632-a0e0-4dfabe363eca None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:07.843978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c550bd3d-7d90-4632-a0e0-4dfabe363eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:07.859510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:07.906860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1281/2531] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:41:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:07.919123 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:07.920750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:07.996287 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725b37c-cd3f-43e1-b059-d31a9c5750c8 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:08.007955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61f9c6-d8f3-42db-8f46-c5bea19e4b58 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:08.008570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f9c6-d8f3-42db-8f46-c5bea19e4b58 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:08.008846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f9c6-d8f3-42db-8f46-c5bea19e4b58 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:08.009129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61f9c6-d8f3-42db-8f46-c5bea19e4b58 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:08.020239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61f9c6-d8f3-42db-8f46-c5bea19e4b58 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:08.023064 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61f9c6-d8f3-42db-8f46-c5bea19e4b58 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:08.040272 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1251/2532] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:41:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:41:17.898903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=16 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:18.035336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:22.032171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156bcb1-7432-4fc7-9a96-ff07846299bd None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:22.033223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156bcb1-7432-4fc7-9a96-ff07846299bd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:22.033688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156bcb1-7432-4fc7-9a96-ff07846299bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:22.062085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b156bcb1-7432-4fc7-9a96-ff07846299bd None None] MFA Rules not processed for user `c11d44667e8e4db5bccccaee16d89a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:22.063687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b156bcb1-7432-4fc7-9a96-ff07846299bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:22.073575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1282/2533] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:23.016726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4fe5-ee3e-4fad-a568-0e7185065a30 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.018124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4fe5-ee3e-4fad-a568-0e7185065a30 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.018815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4fe5-ee3e-4fad-a568-0e7185065a30 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.020997 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1252/2534] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:41:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:41:23.027361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7d8ce-88b0-4962-a792-74914fa652d0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.027665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7d8ce-88b0-4962-a792-74914fa652d0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.027966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7d8ce-88b0-4962-a792-74914fa652d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.054362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4af7d8ce-88b0-4962-a792-74914fa652d0 None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:23.055675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4af7d8ce-88b0-4962-a792-74914fa652d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:23.065681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1283/2535] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:41:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:23.078546 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f4fe5-ee3e-4fad-a568-0e7185065a30 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:23.081359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ee7f4fe5-ee3e-4fad-a568-0e7185065a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:23.153275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f4fe5-ee3e-4fad-a568-0e7185065a30 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:23.158144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e576-1c16-4a72-a6ed-db0b2453b6a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca15f81281dc42f4befeb9335698d18f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:23.158294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e576-1c16-4a72-a6ed-db0b2453b6a5 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.158824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e576-1c16-4a72-a6ed-db0b2453b6a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.159354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e576-1c16-4a72-a6ed-db0b2453b6a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.180568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e576-1c16-4a72-a6ed-db0b2453b6a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:23.182245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e576-1c16-4a72-a6ed-db0b2453b6a5 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:23.200325 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1253/2536] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:23.286518 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9035-ecc1-452a-9ef3-81ee453e9e65 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.286754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9035-ecc1-452a-9ef3-81ee453e9e65 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.287042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9035-ecc1-452a-9ef3-81ee453e9e65 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.288415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1284/2537] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:41:23.294312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc90cc-cd1c-4fd9-a862-3da5a68b09f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.294923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc90cc-cd1c-4fd9-a862-3da5a68b09f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.295250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc90cc-cd1c-4fd9-a862-3da5a68b09f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.318311 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-14dc90cc-cd1c-4fd9-a862-3da5a68b09f0 None None] MFA Rules not processed for user `13603c6cd9544f5391256ea600ad0709`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:23.319694 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-14dc90cc-cd1c-4fd9-a862-3da5a68b09f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:23.376322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1254/2538] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:41:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:23.391723 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a9035-ecc1-452a-9ef3-81ee453e9e65 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:23.394194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6c8a9035-ecc1-452a-9ef3-81ee453e9e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:23.409545 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a9035-ecc1-452a-9ef3-81ee453e9e65 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:23.412994 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b831dc-a9fc-4118-a4c8-62d86bbd7ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:23.413588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b831dc-a9fc-4118-a4c8-62d86bbd7ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.413891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b831dc-a9fc-4118-a4c8-62d86bbd7ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.414161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b831dc-a9fc-4118-a4c8-62d86bbd7ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.421426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b831dc-a9fc-4118-a4c8-62d86bbd7ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:23.422874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b831dc-a9fc-4118-a4c8-62d86bbd7ed3 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:23.430831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1285/2539] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:23.556608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863476-d437-4f97-9e48-4386015702ce None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.556905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863476-d437-4f97-9e48-4386015702ce None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.557172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863476-d437-4f97-9e48-4386015702ce None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.558523 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1255/2540] 10.222.0.42 () {54 vars in 921 bytes} [Wed Aug 31 04:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:41:23.564262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a63e2-9cc2-490a-a39c-9891d7664796 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.565105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a63e2-9cc2-490a-a39c-9891d7664796 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.565105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a63e2-9cc2-490a-a39c-9891d7664796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.587507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-be1a63e2-9cc2-490a-a39c-9891d7664796 None None] MFA Rules not processed for user `708ce6cfc5f64d26a418b4339fd883fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:23.588713 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-be1a63e2-9cc2-490a-a39c-9891d7664796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:23.599692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1286/2541] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Aug 31 04:41:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:23.611350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f863476-d437-4f97-9e48-4386015702ce None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:23.623098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f863476-d437-4f97-9e48-4386015702ce None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:23.627095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829790a-829d-47f6-aa4a-caa0d52729b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:23.627823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829790a-829d-47f6-aa4a-caa0d52729b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:23.628164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829790a-829d-47f6-aa4a-caa0d52729b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:23.628500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829790a-829d-47f6-aa4a-caa0d52729b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:23.639991 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829790a-829d-47f6-aa4a-caa0d52729b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:23.641423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829790a-829d-47f6-aa4a-caa0d52729b7 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:23.652878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1256/2542] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:26.497615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc294d0a-78fe-435c-b47a-a07b999dba4c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:26.498501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc294d0a-78fe-435c-b47a-a07b999dba4c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:26.498816 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc294d0a-78fe-435c-b47a-a07b999dba4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:26.518632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cc294d0a-78fe-435c-b47a-a07b999dba4c None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:26.519913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cc294d0a-78fe-435c-b47a-a07b999dba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:26.527823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1287/2543] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Aug 31 04:41:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:26.539953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829790a-829d-47f6-aa4a-caa0d52729b7 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:26.551543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829790a-829d-47f6-aa4a-caa0d52729b7 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:26.557389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02d7a2-2678-45df-ba7b-78767403a64f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:26.558116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02d7a2-2678-45df-ba7b-78767403a64f service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:26.558401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02d7a2-2678-45df-ba7b-78767403a64f service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:26.558674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02d7a2-2678-45df-ba7b-78767403a64f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:26.568275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02d7a2-2678-45df-ba7b-78767403a64f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:26.569869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02d7a2-2678-45df-ba7b-78767403a64f service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:26.579326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1257/2544] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:41:36.514080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=18 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:36.580326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=61 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:43.991268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b345-d58d-4cd4-9c90-88e273e31b27 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:43.991788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b345-d58d-4cd4-9c90-88e273e31b27 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:43.991788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b345-d58d-4cd4-9c90-88e273e31b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:44.011477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9bb0b345-d58d-4cd4-9c90-88e273e31b27 None None] MFA Rules not processed for user `8381a7bfacbc41b39631c63b29359220`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:44.012703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9bb0b345-d58d-4cd4-9c90-88e273e31b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:44.020582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1288/2545] 10.222.0.42 () {58 vars in 967 bytes} [Wed Aug 31 04:41:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:44.035029 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02d7a2-2678-45df-ba7b-78767403a64f service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:44.046467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02d7a2-2678-45df-ba7b-78767403a64f service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:44.053080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4604fb28234efc9f107cb1f4aa57ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:44.053697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:44.053972 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:44.054207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:44.063673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:44.065258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:44.075627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1258/2546] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Aug 31 04:41:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:41:47.248428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0b345-d58d-4cd4-9c90-88e273e31b27 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:47.259175 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0b345-d58d-4cd4-9c90-88e273e31b27 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:47.263648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:47.264916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:47.265266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:47.265615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:47.267910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:47.333001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:47.334471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:47.344157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1289/2547] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:51.146388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:51.148439 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:51.166569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02559c58-c4da-48a7-80b7-95a74706db3b service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:51.171984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:51.172652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:51.172926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:51.173221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] PATH_INFO: `/v3/users/718418f954f94dad93eeb725e345c850` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:51.174339 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=718418f954f94dad93eeb725e345c850)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:51.176424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:51.224746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '718418f954f94dad93eeb725e345c850'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.232584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '718418f954f94dad93eeb725e345c850'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.233677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '718418f954f94dad93eeb725e345c850'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.246171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '718418f954f94dad93eeb725e345c850'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.246982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-571e9c80-1edc-4491-8f05-47e07001c361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '718418f954f94dad93eeb725e345c850'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.259957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1259/2548] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:41:51 2022] DELETE /identity/v3/users/718418f954f94dad93eeb725e345c850 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:51.438831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:51.455295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22c274-d428-4d54-99f7-32cdb6519d87 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:51.527693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:51.528373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:51.529301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:51.529580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] PATH_INFO: `/v3/projects/94afda48b8e545dfaeac33a66f592fe2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:51.530706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=94afda48b8e545dfaeac33a66f592fe2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:51.532649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:51.582383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94afda48b8e545dfaeac33a66f592fe2', 'request_id': 'req-d223eb16-62f1-4a4d-ae28-d33338556a9f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.591575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94afda48b8e545dfaeac33a66f592fe2', 'request_id': 'req-d223eb16-62f1-4a4d-ae28-d33338556a9f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.598421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94afda48b8e545dfaeac33a66f592fe2', 'request_id': 'req-d223eb16-62f1-4a4d-ae28-d33338556a9f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.610984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] The token cache is being invalidate because project 94afda48b8e545dfaeac33a66f592fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:41:51.611272 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.611971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d223eb16-62f1-4a4d-ae28-d33338556a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:51.613499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1290/2549] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:41:51 2022] DELETE /identity/v3/projects/94afda48b8e545dfaeac33a66f592fe2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:51.618827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb944-1f58-4e51-a86a-784e8c3e3894 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:51.619117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb944-1f58-4e51-a86a-784e8c3e3894 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:51.619393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb944-1f58-4e51-a86a-784e8c3e3894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:51.640459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ca7bb944-1f58-4e51-a86a-784e8c3e3894 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:51.641674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca7bb944-1f58-4e51-a86a-784e8c3e3894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:51.703969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1260/2550] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:51.712267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dd1b5-4fb1-47cc-8a83-513aa0ac5352 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:51.712566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dd1b5-4fb1-47cc-8a83-513aa0ac5352 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:51.712829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dd1b5-4fb1-47cc-8a83-513aa0ac5352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:51.740360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3a6dd1b5-4fb1-47cc-8a83-513aa0ac5352 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:51.742207 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3a6dd1b5-4fb1-47cc-8a83-513aa0ac5352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:51.751137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1291/2551] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:51.756168 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bb944-1f58-4e51-a86a-784e8c3e3894 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:51.766403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bb944-1f58-4e51-a86a-784e8c3e3894 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:51.771840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:51.772516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:51.772794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:51.773066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:51.773719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:41:51.774023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:51.775665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:51.787709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1261/2552] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:41:51.792276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dd1b5-4fb1-47cc-8a83-513aa0ac5352 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:51.801831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dd1b5-4fb1-47cc-8a83-513aa0ac5352 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:51.807447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39b40-ba7c-4127-ad97-6c70d51dfcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:51.808071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39b40-ba7c-4127-ad97-6c70d51dfcca admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:51.808344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39b40-ba7c-4127-ad97-6c70d51dfcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:51.808624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a39b40-ba7c-4127-ad97-6c70d51dfcca admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:51.809289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a39b40-ba7c-4127-ad97-6c70d51dfcca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:51.811239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a39b40-ba7c-4127-ad97-6c70d51dfcca admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:51.821343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:51.842919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1292/2553] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:41:51 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:41:51.847004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:51.857643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6687e3-064e-4d2c-889b-27bb97aefc3c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:51.905653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:51.925115 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e586b-abdc-4425-abb9-3b04db694d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:51.925115 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e586b-abdc-4425-abb9-3b04db694d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:51.925408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e586b-abdc-4425-abb9-3b04db694d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:51.925518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e586b-abdc-4425-abb9-3b04db694d2d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:51.926077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e586b-abdc-4425-abb9-3b04db694d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:51.928156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e586b-abdc-4425-abb9-3b04db694d2d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:51.957309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1262/2554] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:41:51 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:41:51.958614 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39b40-ba7c-4127-ad97-6c70d51dfcca admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:51.969400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a39b40-ba7c-4127-ad97-6c70d51dfcca admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:51.974009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7526f92-2353-452c-a5ac-063ab0f1d894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:51.974627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526f92-2353-452c-a5ac-063ab0f1d894 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:51.974933 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526f92-2353-452c-a5ac-063ab0f1d894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:51.975206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526f92-2353-452c-a5ac-063ab0f1d894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:51.976162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7526f92-2353-452c-a5ac-063ab0f1d894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:51.977771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7526f92-2353-452c-a5ac-063ab0f1d894 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:51.988570 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1293/2555] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:41:51.993107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e586b-abdc-4425-abb9-3b04db694d2d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:52.004894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e586b-abdc-4425-abb9-3b04db694d2d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:52.010046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaae002-ed1c-4608-9c1b-caf44411bd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:52.010663 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaae002-ed1c-4608-9c1b-caf44411bd77 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:52.010977 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaae002-ed1c-4608-9c1b-caf44411bd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:52.011226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaae002-ed1c-4608-9c1b-caf44411bd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:52.011907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaae002-ed1c-4608-9c1b-caf44411bd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:52.013526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaae002-ed1c-4608-9c1b-caf44411bd77 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:52.020465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689cfa-48e5-4164-ad0d-bdb52fa3c98a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:52.020868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689cfa-48e5-4164-ad0d-bdb52fa3c98a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:52.021227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689cfa-48e5-4164-ad0d-bdb52fa3c98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:52.031291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1263/2556] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:41:52.034060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaae002-ed1c-4608-9c1b-caf44411bd77 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:52.052147 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaae002-ed1c-4608-9c1b-caf44411bd77 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:52.054238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-19689cfa-48e5-4164-ad0d-bdb52fa3c98a None None] MFA Rules not processed for user `9f4dcda6ad1d401a9aed637906563d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:52.055777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-19689cfa-48e5-4164-ad0d-bdb52fa3c98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:52.067569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8dc61-1df3-4bd1-a4f5-002982c9eac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:52.068649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8dc61-1df3-4bd1-a4f5-002982c9eac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:52.071218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8dc61-1df3-4bd1-a4f5-002982c9eac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:52.075038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8dc61-1df3-4bd1-a4f5-002982c9eac3 admin admin] PATH_INFO: `/v3/projects/bd6a276a007c4dce92019fffbc54c80a/users/aa7eb8e35a114c1c97f9ba9851126882/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:52.094271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8dc61-1df3-4bd1-a4f5-002982c9eac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6a276a007c4dce92019fffbc54c80a, user_id=aa7eb8e35a114c1c97f9ba9851126882, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:52.098246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8dc61-1df3-4bd1-a4f5-002982c9eac3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:52.132670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1264/2557] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:41:52 2022] PUT /identity/v3/projects/bd6a276a007c4dce92019fffbc54c80a/users/aa7eb8e35a114c1c97f9ba9851126882/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:52.142995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499a4b2-ffb4-474d-be09-d0c2bcc80815 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:52.143246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499a4b2-ffb4-474d-be09-d0c2bcc80815 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:52.143501 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499a4b2-ffb4-474d-be09-d0c2bcc80815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:52.158359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1294/2558] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:52.179210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4499a4b2-ffb4-474d-be09-d0c2bcc80815 None None] MFA Rules not processed for user `aa7eb8e35a114c1c97f9ba9851126882`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:52.180433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4499a4b2-ffb4-474d-be09-d0c2bcc80815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:52.258505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1265/2559] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:41:52.269730 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19689cfa-48e5-4164-ad0d-bdb52fa3c98a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:52.271360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-19689cfa-48e5-4164-ad0d-bdb52fa3c98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:52.356613 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19689cfa-48e5-4164-ad0d-bdb52fa3c98a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:52.423559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f10f4a-266b-4877-ab92-abbef5b7c85c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:52.424224 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f10f4a-266b-4877-ab92-abbef5b7c85c service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:52.424450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f10f4a-266b-4877-ab92-abbef5b7c85c service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:52.424720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f10f4a-266b-4877-ab92-abbef5b7c85c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:52.436890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f10f4a-266b-4877-ab92-abbef5b7c85c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:52.438522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f10f4a-266b-4877-ab92-abbef5b7c85c service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:52.449249 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1295/2560] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:53.483890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499a4b2-ffb4-474d-be09-d0c2bcc80815 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:53.486102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4499a4b2-ffb4-474d-be09-d0c2bcc80815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:53.550342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499a4b2-ffb4-474d-be09-d0c2bcc80815 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:53.554156 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e34256-2d63-44a5-b785-2fa26c4edf0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:53.554806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e34256-2d63-44a5-b785-2fa26c4edf0b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:53.555081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e34256-2d63-44a5-b785-2fa26c4edf0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:53.555345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e34256-2d63-44a5-b785-2fa26c4edf0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:53.557181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e8e34256-2d63-44a5-b785-2fa26c4edf0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:53.602018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:53.621474 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e34256-2d63-44a5-b785-2fa26c4edf0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:53.622941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e34256-2d63-44a5-b785-2fa26c4edf0b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:53.633146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1266/2561] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:55.195064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f10f4a-266b-4877-ab92-abbef5b7c85c service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:55.208502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:55.278546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f10f4a-266b-4877-ab92-abbef5b7c85c service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:55.285152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:55.286208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:55.287058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:55.287637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] PATH_INFO: `/v3/users/aa7eb8e35a114c1c97f9ba9851126882` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:55.290326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7eb8e35a114c1c97f9ba9851126882)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:55.293300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:55.361214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7eb8e35a114c1c97f9ba9851126882'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:55.369299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7eb8e35a114c1c97f9ba9851126882'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:55.371137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7eb8e35a114c1c97f9ba9851126882'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:55.384663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7eb8e35a114c1c97f9ba9851126882'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:55.385401 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7eb8e35a114c1c97f9ba9851126882'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:55.399390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1296/2562] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:41:55 2022] DELETE /identity/v3/users/aa7eb8e35a114c1c97f9ba9851126882 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:55.403648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed69a94-0bca-4e0b-ba67-60aa2bbdc5d2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:55.404149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed69a94-0bca-4e0b-ba67-60aa2bbdc5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:55.404233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed69a94-0bca-4e0b-ba67-60aa2bbdc5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:55.423923 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fed69a94-0bca-4e0b-ba67-60aa2bbdc5d2 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:55.425146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fed69a94-0bca-4e0b-ba67-60aa2bbdc5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:55.496344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1267/2563] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:55.508091 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:55.509821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:55.582802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ff24-b34d-4d2a-b386-1f3343d2c677 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:55.588406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b9e03-f6b3-478e-8ddf-b4e656f0382b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:55.589046 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b9e03-f6b3-478e-8ddf-b4e656f0382b service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:55.589376 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b9e03-f6b3-478e-8ddf-b4e656f0382b service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:55.589654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b9e03-f6b3-478e-8ddf-b4e656f0382b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:55.601551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b9e03-f6b3-478e-8ddf-b4e656f0382b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:55.604522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b9e03-f6b3-478e-8ddf-b4e656f0382b service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:55.630317 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1297/2564] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:55.957757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed69a94-0bca-4e0b-ba67-60aa2bbdc5d2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:55.959493 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fed69a94-0bca-4e0b-ba67-60aa2bbdc5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:55.982766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed69a94-0bca-4e0b-ba67-60aa2bbdc5d2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:55.991756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:55.992605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:55.992970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:55.993442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] PATH_INFO: `/v3/projects/bd6a276a007c4dce92019fffbc54c80a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:55.994918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6a276a007c4dce92019fffbc54c80a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:56.000551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:56.068579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6a276a007c4dce92019fffbc54c80a', 'request_id': 'req-384cbec3-7782-405c-a977-6448912c38b0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:56.079718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6a276a007c4dce92019fffbc54c80a', 'request_id': 'req-384cbec3-7782-405c-a977-6448912c38b0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:56.094452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6a276a007c4dce92019fffbc54c80a', 'request_id': 'req-384cbec3-7782-405c-a977-6448912c38b0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:56.109669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] The token cache is being invalidate because project bd6a276a007c4dce92019fffbc54c80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:41:56.110021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:56.110671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-384cbec3-7782-405c-a977-6448912c38b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:56.114796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1268/2565] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:41:55 2022] DELETE /identity/v3/projects/bd6a276a007c4dce92019fffbc54c80a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:56.119459 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedbde8-b70b-4066-8674-c06bb7239738 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.120607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedbde8-b70b-4066-8674-c06bb7239738 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.120607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedbde8-b70b-4066-8674-c06bb7239738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.140768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8fedbde8-b70b-4066-8674-c06bb7239738 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:56.141997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8fedbde8-b70b-4066-8674-c06bb7239738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:56.169495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:56.209463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1298/2566] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:41:56.217791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f61a77-055f-4e54-8281-16ef5b948c64 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.218078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f61a77-055f-4e54-8281-16ef5b948c64 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.218349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f61a77-055f-4e54-8281-16ef5b948c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.241008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-34f61a77-055f-4e54-8281-16ef5b948c64 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:56.242299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34f61a77-055f-4e54-8281-16ef5b948c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:56.251944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1269/2567] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:56.256009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedbde8-b70b-4066-8674-c06bb7239738 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:56.266855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedbde8-b70b-4066-8674-c06bb7239738 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:56.271547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:56.272167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.272505 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.272775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.273430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:41:56.273725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:56.275400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:56.288279 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1299/2568] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:41:56.292392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f61a77-055f-4e54-8281-16ef5b948c64 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:56.302047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f61a77-055f-4e54-8281-16ef5b948c64 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:56.307833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec68fc-b0d3-46fb-9e31-0e6be42dd0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:56.308591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec68fc-b0d3-46fb-9e31-0e6be42dd0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.308862 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec68fc-b0d3-46fb-9e31-0e6be42dd0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.309139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec68fc-b0d3-46fb-9e31-0e6be42dd0d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.310076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec68fc-b0d3-46fb-9e31-0e6be42dd0d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:56.311901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec68fc-b0d3-46fb-9e31-0e6be42dd0d3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:56.348496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1270/2569] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:41:56 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:41:56.353875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:56.367340 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d28678-2167-41fd-9439-8674b1bfa57e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:56.476623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d45d7-fa3d-48e8-80ab-5f7dee9fa0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:56.477632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d45d7-fa3d-48e8-80ab-5f7dee9fa0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.478157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d45d7-fa3d-48e8-80ab-5f7dee9fa0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.478654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d45d7-fa3d-48e8-80ab-5f7dee9fa0a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.479883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d45d7-fa3d-48e8-80ab-5f7dee9fa0a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:56.482074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d45d7-fa3d-48e8-80ab-5f7dee9fa0a4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:56.522851 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1300/2570] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:41:56 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:41:56.524107 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec68fc-b0d3-46fb-9e31-0e6be42dd0d3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:56.535568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec68fc-b0d3-46fb-9e31-0e6be42dd0d3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:56.541934 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e75e1b-53d7-472e-920f-1f45e92e27d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:56.543002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e75e1b-53d7-472e-920f-1f45e92e27d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.544035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e75e1b-53d7-472e-920f-1f45e92e27d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.544604 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e75e1b-53d7-472e-920f-1f45e92e27d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.551418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e75e1b-53d7-472e-920f-1f45e92e27d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:56.553182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e75e1b-53d7-472e-920f-1f45e92e27d8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:56.566943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1271/2571] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:41:56.571306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d45d7-fa3d-48e8-80ab-5f7dee9fa0a4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:56.588136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d45d7-fa3d-48e8-80ab-5f7dee9fa0a4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:56.594691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad1516-e00d-4f85-b6f1-f7b018b02501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:56.595367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad1516-e00d-4f85-b6f1-f7b018b02501 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.595652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad1516-e00d-4f85-b6f1-f7b018b02501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.595931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad1516-e00d-4f85-b6f1-f7b018b02501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.596600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad1516-e00d-4f85-b6f1-f7b018b02501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:56.598229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad1516-e00d-4f85-b6f1-f7b018b02501 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:56.609923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1301/2572] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:41:56.614067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e75e1b-53d7-472e-920f-1f45e92e27d8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:56.624719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e75e1b-53d7-472e-920f-1f45e92e27d8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:56.629525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:56.630556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.630949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.631125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] PATH_INFO: `/v3/projects/640529f2263b41b7aca6944eec599c8e/users/c5aef741693d4bceb88359b6b6f4815d/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.651635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=640529f2263b41b7aca6944eec599c8e, user_id=c5aef741693d4bceb88359b6b6f4815d, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:56.654303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:56.666152 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:56.673640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1272/2573] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:41:56 2022] PUT /identity/v3/projects/640529f2263b41b7aca6944eec599c8e/users/c5aef741693d4bceb88359b6b6f4815d/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:56.683957 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f5a1-7b74-437b-b9ab-4dce409eadf7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.684353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f5a1-7b74-437b-b9ab-4dce409eadf7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.684732 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f5a1-7b74-437b-b9ab-4dce409eadf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.719371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0564f5a1-7b74-437b-b9ab-4dce409eadf7 None None] MFA Rules not processed for user `c5aef741693d4bceb88359b6b6f4815d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:56.721584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0564f5a1-7b74-437b-b9ab-4dce409eadf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:56.808810 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1302/2574] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:56.821309 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:56.823464 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:56.929353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782dc19-4633-460b-949e-4163c0fbb694 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:56.938829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f95ae-4d31-4b0b-aeea-e524e176a907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:56.940001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f95ae-4d31-4b0b-aeea-e524e176a907 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:56.940469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f95ae-4d31-4b0b-aeea-e524e176a907 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:56.940735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f95ae-4d31-4b0b-aeea-e524e176a907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:56.949433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f95ae-4d31-4b0b-aeea-e524e176a907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:56.951169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f95ae-4d31-4b0b-aeea-e524e176a907 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:56.960897 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1273/2575] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:41:57.180147 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564f5a1-7b74-437b-b9ab-4dce409eadf7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:57.210301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:57.247616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564f5a1-7b74-437b-b9ab-4dce409eadf7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:57.252658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:57.253300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:57.253575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:57.253838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] PATH_INFO: `/v3/users/c5aef741693d4bceb88359b6b6f4815d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:57.255083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5aef741693d4bceb88359b6b6f4815d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:57.257113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:57.310581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5aef741693d4bceb88359b6b6f4815d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.319749 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5aef741693d4bceb88359b6b6f4815d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.320626 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5aef741693d4bceb88359b6b6f4815d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.333166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5aef741693d4bceb88359b6b6f4815d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.333844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5aef741693d4bceb88359b6b6f4815d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.346466 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1303/2576] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:41:57 2022] DELETE /identity/v3/users/c5aef741693d4bceb88359b6b6f4815d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:57.352590 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec5bd2-9b8e-44d3-a93f-8596d894ddca None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:57.352876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec5bd2-9b8e-44d3-a93f-8596d894ddca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:57.353145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec5bd2-9b8e-44d3-a93f-8596d894ddca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:57.374284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b7ec5bd2-9b8e-44d3-a93f-8596d894ddca None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:57.375520 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7ec5bd2-9b8e-44d3-a93f-8596d894ddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:57.435891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1274/2577] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:57.447774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:57.449663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:57.516389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890113-ab83-429c-a193-fd82b4808ca0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:57.520366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde2bb4-da2c-4c21-8ff8-fed3e332ca5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:57.520971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde2bb4-da2c-4c21-8ff8-fed3e332ca5d service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:57.521235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde2bb4-da2c-4c21-8ff8-fed3e332ca5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:57.521503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde2bb4-da2c-4c21-8ff8-fed3e332ca5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:57.530637 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde2bb4-da2c-4c21-8ff8-fed3e332ca5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:57.532243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde2bb4-da2c-4c21-8ff8-fed3e332ca5d service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:57.541634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1304/2578] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:57.880617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec5bd2-9b8e-44d3-a93f-8596d894ddca None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:57.882263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7ec5bd2-9b8e-44d3-a93f-8596d894ddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:57.894551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec5bd2-9b8e-44d3-a93f-8596d894ddca None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:57.899913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:57.900555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:57.900828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:57.901095 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] PATH_INFO: `/v3/projects/640529f2263b41b7aca6944eec599c8e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:57.902228 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=640529f2263b41b7aca6944eec599c8e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:57.904179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:57.954337 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640529f2263b41b7aca6944eec599c8e', 'request_id': 'req-e3f34269-d87f-4302-b31c-43b53f852f7e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.961038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640529f2263b41b7aca6944eec599c8e', 'request_id': 'req-e3f34269-d87f-4302-b31c-43b53f852f7e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.968950 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640529f2263b41b7aca6944eec599c8e', 'request_id': 'req-e3f34269-d87f-4302-b31c-43b53f852f7e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.979308 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] The token cache is being invalidate because project 640529f2263b41b7aca6944eec599c8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:41:57.979592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.980217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e3f34269-d87f-4302-b31c-43b53f852f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:57.981714 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1275/2579] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:41:57 2022] DELETE /identity/v3/projects/640529f2263b41b7aca6944eec599c8e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:57.987192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a1e8f-1256-432f-859e-5279457472fa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:57.987477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a1e8f-1256-432f-859e-5279457472fa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:57.987747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a1e8f-1256-432f-859e-5279457472fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.008045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-213a1e8f-1256-432f-859e-5279457472fa None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:58.009240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-213a1e8f-1256-432f-859e-5279457472fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:58.052069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:58.072710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1305/2580] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:58.081000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfe41f-9480-4a91-b3d8-89bee09985c8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.081275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfe41f-9480-4a91-b3d8-89bee09985c8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.081539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfe41f-9480-4a91-b3d8-89bee09985c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.101995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-76dfe41f-9480-4a91-b3d8-89bee09985c8 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:58.103209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-76dfe41f-9480-4a91-b3d8-89bee09985c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:58.112318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1276/2581] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:58.116887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a1e8f-1256-432f-859e-5279457472fa None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.127104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a1e8f-1256-432f-859e-5279457472fa None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.132855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.133747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.134025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.134359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.135316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:41:58.135775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.138242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.154534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1306/2582] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:41:58.159297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfe41f-9480-4a91-b3d8-89bee09985c8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.163966 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:58.172266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfe41f-9480-4a91-b3d8-89bee09985c8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.178806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd0be9-9b3d-4d7c-aad3-cfa860ce9b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.179503 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd0be9-9b3d-4d7c-aad3-cfa860ce9b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.179868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd0be9-9b3d-4d7c-aad3-cfa860ce9b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.180241 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd0be9-9b3d-4d7c-aad3-cfa860ce9b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.180973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd0be9-9b3d-4d7c-aad3-cfa860ce9b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.183563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd0be9-9b3d-4d7c-aad3-cfa860ce9b33 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.220799 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1277/2583] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:41:58 2022] POST /identity/v3/projects => generated 363 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:41:58.225061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.235967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96e01d-9e13-428d-ad2c-3e93c7de4f51 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.294651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157ab48-92ce-481a-aec2-f2f2b8a6c451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.295305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ab48-92ce-481a-aec2-f2f2b8a6c451 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.295575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ab48-92ce-481a-aec2-f2f2b8a6c451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.295865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ab48-92ce-481a-aec2-f2f2b8a6c451 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.296501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157ab48-92ce-481a-aec2-f2f2b8a6c451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.298417 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157ab48-92ce-481a-aec2-f2f2b8a6c451 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.322498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1307/2584] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:41:58 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:41:58.326576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd0be9-9b3d-4d7c-aad3-cfa860ce9b33 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.337108 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd0be9-9b3d-4d7c-aad3-cfa860ce9b33 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.341728 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d86548-f5a4-41fa-8e2c-dd51772d9d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.342333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d86548-f5a4-41fa-8e2c-dd51772d9d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.342605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d86548-f5a4-41fa-8e2c-dd51772d9d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.343035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d86548-f5a4-41fa-8e2c-dd51772d9d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.343704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d86548-f5a4-41fa-8e2c-dd51772d9d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.345301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d86548-f5a4-41fa-8e2c-dd51772d9d07 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.355778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1278/2585] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:41:58.360349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157ab48-92ce-481a-aec2-f2f2b8a6c451 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.370621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157ab48-92ce-481a-aec2-f2f2b8a6c451 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.375526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b33c9-e8ec-4706-b2d5-51e8b4cce658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.376166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b33c9-e8ec-4706-b2d5-51e8b4cce658 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.376440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b33c9-e8ec-4706-b2d5-51e8b4cce658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.376705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b33c9-e8ec-4706-b2d5-51e8b4cce658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.377344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b33c9-e8ec-4706-b2d5-51e8b4cce658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.378982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b33c9-e8ec-4706-b2d5-51e8b4cce658 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.389656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1308/2586] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:41:58.393899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d86548-f5a4-41fa-8e2c-dd51772d9d07 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.403901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d86548-f5a4-41fa-8e2c-dd51772d9d07 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.408627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef090e-5609-4702-8e54-ae9351d58b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.409295 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef090e-5609-4702-8e54-ae9351d58b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.409588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef090e-5609-4702-8e54-ae9351d58b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.409855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef090e-5609-4702-8e54-ae9351d58b38 admin admin] PATH_INFO: `/v3/projects/7a8f285128734860a051deae3e4bb1fe/users/70f4182f3b984fff8137784b25519d2d/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.425069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef090e-5609-4702-8e54-ae9351d58b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8f285128734860a051deae3e4bb1fe, user_id=70f4182f3b984fff8137784b25519d2d, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.427745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef090e-5609-4702-8e54-ae9351d58b38 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.443163 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1279/2587] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:41:58 2022] PUT /identity/v3/projects/7a8f285128734860a051deae3e4bb1fe/users/70f4182f3b984fff8137784b25519d2d/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:41:58.453599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcd279-be00-4252-8898-32860a597568 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.453882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcd279-be00-4252-8898-32860a597568 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.454154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcd279-be00-4252-8898-32860a597568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.484598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7ebcd279-be00-4252-8898-32860a597568 None None] MFA Rules not processed for user `70f4182f3b984fff8137784b25519d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:58.485845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7ebcd279-be00-4252-8898-32860a597568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:58.550088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1309/2588] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:58.554794 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef090e-5609-4702-8e54-ae9351d58b38 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.619738 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef090e-5609-4702-8e54-ae9351d58b38 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.624705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92701f0-e597-4dbe-b0f4-0b2549602e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.625391 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92701f0-e597-4dbe-b0f4-0b2549602e3c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.625674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92701f0-e597-4dbe-b0f4-0b2549602e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.625942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92701f0-e597-4dbe-b0f4-0b2549602e3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.626579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92701f0-e597-4dbe-b0f4-0b2549602e3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.628527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92701f0-e597-4dbe-b0f4-0b2549602e3c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.659795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1280/2589] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:41:58 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:41:58.661412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcd279-be00-4252-8898-32860a597568 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.673006 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcd279-be00-4252-8898-32860a597568 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.715455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:58.735015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355a6b1-83ae-4ed6-af75-7396e9ebd65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.735635 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355a6b1-83ae-4ed6-af75-7396e9ebd65c admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.735914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355a6b1-83ae-4ed6-af75-7396e9ebd65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.736182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355a6b1-83ae-4ed6-af75-7396e9ebd65c admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.736833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355a6b1-83ae-4ed6-af75-7396e9ebd65c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.738791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355a6b1-83ae-4ed6-af75-7396e9ebd65c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.761661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1310/2590] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:41:58 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:41:58.765832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92701f0-e597-4dbe-b0f4-0b2549602e3c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.776351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92701f0-e597-4dbe-b0f4-0b2549602e3c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.781381 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd56a55-0415-4b71-ba76-04aade5445ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.781982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56a55-0415-4b71-ba76-04aade5445ea admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.782268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56a55-0415-4b71-ba76-04aade5445ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.782537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56a55-0415-4b71-ba76-04aade5445ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.783216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd56a55-0415-4b71-ba76-04aade5445ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.784875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd56a55-0415-4b71-ba76-04aade5445ea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.796707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1281/2591] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:41:58.800910 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355a6b1-83ae-4ed6-af75-7396e9ebd65c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.811359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355a6b1-83ae-4ed6-af75-7396e9ebd65c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.816218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf16b9a-7d4f-4712-b2d2-5773ada665b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.816880 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16b9a-7d4f-4712-b2d2-5773ada665b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.817158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16b9a-7d4f-4712-b2d2-5773ada665b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.817433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16b9a-7d4f-4712-b2d2-5773ada665b8 admin admin] PATH_INFO: `/v3/projects/9644468bd79547758aeef6b10837484f/users/9d21891f42a4434a8863f0f540609860/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.833290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf16b9a-7d4f-4712-b2d2-5773ada665b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9644468bd79547758aeef6b10837484f, user_id=9d21891f42a4434a8863f0f540609860, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.836035 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf16b9a-7d4f-4712-b2d2-5773ada665b8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.851385 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1311/2592] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:41:58 2022] PUT /identity/v3/projects/9644468bd79547758aeef6b10837484f/users/9d21891f42a4434a8863f0f540609860/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:58.856102 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd56a55-0415-4b71-ba76-04aade5445ea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.866790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd56a55-0415-4b71-ba76-04aade5445ea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.925247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba6b4a-6f7f-49f1-9c3b-b1386464fcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.925949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba6b4a-6f7f-49f1-9c3b-b1386464fcfe admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.926201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba6b4a-6f7f-49f1-9c3b-b1386464fcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.926467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba6b4a-6f7f-49f1-9c3b-b1386464fcfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.927174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba6b4a-6f7f-49f1-9c3b-b1386464fcfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.928821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba6b4a-6f7f-49f1-9c3b-b1386464fcfe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.942055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1282/2593] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:41:58.943662 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf16b9a-7d4f-4712-b2d2-5773ada665b8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:58.954513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf16b9a-7d4f-4712-b2d2-5773ada665b8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:58.959109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:58.959770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.960114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.960377 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] PATH_INFO: `/v3/projects/9644468bd79547758aeef6b10837484f/users/9d21891f42a4434a8863f0f540609860/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:58.962276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9644468bd79547758aeef6b10837484f, user_id=9d21891f42a4434a8863f0f540609860, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:58.964930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:58.975515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1312/2594] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:41:58 2022] PUT /identity/v3/projects/9644468bd79547758aeef6b10837484f/users/9d21891f42a4434a8863f0f540609860/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:58.987755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6ed1c-16b2-4a90-a620-dac2e7349f3e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:58.988038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6ed1c-16b2-4a90-a620-dac2e7349f3e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:58.988306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6ed1c-16b2-4a90-a620-dac2e7349f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:59.016251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e6b6ed1c-16b2-4a90-a620-dac2e7349f3e None None] MFA Rules not processed for user `9d21891f42a4434a8863f0f540609860`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:59.017476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e6b6ed1c-16b2-4a90-a620-dac2e7349f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:59.021433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:59.086440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1283/2595] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:59.097806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:59.099659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:59.157036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af395b2c-964c-4fed-b9de-27611e043559 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:59.161032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c666494-98d2-4675-a62d-7e47eb9f54ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:59.161632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c666494-98d2-4675-a62d-7e47eb9f54ce service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:59.161908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c666494-98d2-4675-a62d-7e47eb9f54ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:59.162206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c666494-98d2-4675-a62d-7e47eb9f54ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:59.171245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c666494-98d2-4675-a62d-7e47eb9f54ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:59.172680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c666494-98d2-4675-a62d-7e47eb9f54ce service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:59.181847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1313/2596] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:41:59.635097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6ed1c-16b2-4a90-a620-dac2e7349f3e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:59.699373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6ed1c-16b2-4a90-a620-dac2e7349f3e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:59.704793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:59.705427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:59.705710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:59.705979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] PATH_INFO: `/v3/users/70f4182f3b984fff8137784b25519d2d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:59.707324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f4182f3b984fff8137784b25519d2d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:59.709319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:41:59.758965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f4182f3b984fff8137784b25519d2d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:59.767514 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f4182f3b984fff8137784b25519d2d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:59.768139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f4182f3b984fff8137784b25519d2d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:59.780562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f4182f3b984fff8137784b25519d2d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:59.781191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f4182f3b984fff8137784b25519d2d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:41:59.793409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1284/2597] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:41:59 2022] DELETE /identity/v3/users/70f4182f3b984fff8137784b25519d2d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:41:59.797992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e5d5-e95e-47b2-90d0-ef9ca73a53d1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:59.798450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e5d5-e95e-47b2-90d0-ef9ca73a53d1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:59.798546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e5d5-e95e-47b2-90d0-ef9ca73a53d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:59.821190 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2ff8e5d5-e95e-47b2-90d0-ef9ca73a53d1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:41:59.822462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2ff8e5d5-e95e-47b2-90d0-ef9ca73a53d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:59.892824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1314/2598] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:41:59.903013 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:41:59.904497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:41:59.927511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:41:59.971624 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114d2b1-eca2-4346-9a39-0670b5432518 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:41:59.976074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565ae4f-c342-483d-b577-6cc9b21741a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:41:59.976681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ae4f-c342-483d-b577-6cc9b21741a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:41:59.976951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ae4f-c342-483d-b577-6cc9b21741a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:41:59.977211 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ae4f-c342-483d-b577-6cc9b21741a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:41:59.990805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565ae4f-c342-483d-b577-6cc9b21741a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:41:59.992731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565ae4f-c342-483d-b577-6cc9b21741a5 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:00.002506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1285/2599] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:00.272489 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8e5d5-e95e-47b2-90d0-ef9ca73a53d1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:00.274915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2ff8e5d5-e95e-47b2-90d0-ef9ca73a53d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:00.287944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8e5d5-e95e-47b2-90d0-ef9ca73a53d1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:00.294343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:00.295111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:00.295383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:00.295636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] PATH_INFO: `/v3/projects/7a8f285128734860a051deae3e4bb1fe` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:00.297041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8f285128734860a051deae3e4bb1fe)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:00.299069 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:00.339295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:00.354650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8f285128734860a051deae3e4bb1fe', 'request_id': 'req-526c177e-6702-49a5-b7ef-430c4f3fe62f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.363456 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8f285128734860a051deae3e4bb1fe', 'request_id': 'req-526c177e-6702-49a5-b7ef-430c4f3fe62f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.370316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8f285128734860a051deae3e4bb1fe', 'request_id': 'req-526c177e-6702-49a5-b7ef-430c4f3fe62f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.382736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] The token cache is being invalidate because project 7a8f285128734860a051deae3e4bb1fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:00.383016 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.383668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.385177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1315/2600] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:00 2022] DELETE /identity/v3/projects/7a8f285128734860a051deae3e4bb1fe => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:00.389469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565ae4f-c342-483d-b577-6cc9b21741a5 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:00.399417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565ae4f-c342-483d-b577-6cc9b21741a5 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:00.457468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:00.458099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:00.458373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:00.458675 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] PATH_INFO: `/v3/users/9d21891f42a4434a8863f0f540609860` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:00.459882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d21891f42a4434a8863f0f540609860)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:00.461990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:00.513601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d21891f42a4434a8863f0f540609860'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.520576 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d21891f42a4434a8863f0f540609860'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.521276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d21891f42a4434a8863f0f540609860'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.533099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d21891f42a4434a8863f0f540609860'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.533801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6300ed84-6867-4e2b-b9c3-2d1619288bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d21891f42a4434a8863f0f540609860'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.547343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1286/2601] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:00 2022] DELETE /identity/v3/users/9d21891f42a4434a8863f0f540609860 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:42:00.827628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:00.829601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:00.845666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c177e-6702-49a5-b7ef-430c4f3fe62f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:00.907533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:00.908178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:00.908454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:00.908741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] PATH_INFO: `/v3/projects/9644468bd79547758aeef6b10837484f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:00.909919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9644468bd79547758aeef6b10837484f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:00.911893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:00.961425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9644468bd79547758aeef6b10837484f', 'request_id': 'req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.970307 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9644468bd79547758aeef6b10837484f', 'request_id': 'req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.977294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9644468bd79547758aeef6b10837484f', 'request_id': 'req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.988622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] The token cache is being invalidate because project 9644468bd79547758aeef6b10837484f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:00.988877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.989572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eae8b5f5-cd5d-4353-a50e-8eeb206fe2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:00.991201 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1316/2602] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:00 2022] DELETE /identity/v3/projects/9644468bd79547758aeef6b10837484f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:00.996954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff10d1-9d18-422f-af16-c03f7f14972b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:00.996954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff10d1-9d18-422f-af16-c03f7f14972b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:00.997757 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff10d1-9d18-422f-af16-c03f7f14972b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.025080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d5ff10d1-9d18-422f-af16-c03f7f14972b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:01.026303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d5ff10d1-9d18-422f-af16-c03f7f14972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:01.078882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:01.105734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1287/2603] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:01.114109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9584c-2031-4bd7-b7d2-32ffa62de8fe None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.114389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9584c-2031-4bd7-b7d2-32ffa62de8fe None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.114663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9584c-2031-4bd7-b7d2-32ffa62de8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.135724 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f9b9584c-2031-4bd7-b7d2-32ffa62de8fe None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:01.136950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f9b9584c-2031-4bd7-b7d2-32ffa62de8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:01.145002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1317/2604] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:42:01.150103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff10d1-9d18-422f-af16-c03f7f14972b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.159596 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff10d1-9d18-422f-af16-c03f7f14972b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.164949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.165687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.166083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.166626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.167390 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:42:01.167756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.169417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.182197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1288/2605] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:42:01.186454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9584c-2031-4bd7-b7d2-32ffa62de8fe None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.196552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9584c-2031-4bd7-b7d2-32ffa62de8fe None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.201807 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e49e69-31cb-4f58-afc8-5e7a7f5bda09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.202428 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e49e69-31cb-4f58-afc8-5e7a7f5bda09 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.203243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e49e69-31cb-4f58-afc8-5e7a7f5bda09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.203602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e49e69-31cb-4f58-afc8-5e7a7f5bda09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.204275 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e49e69-31cb-4f58-afc8-5e7a7f5bda09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.206198 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e49e69-31cb-4f58-afc8-5e7a7f5bda09 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.238217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1318/2606] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:01 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:01.242456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.254358 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc76d3-3b2d-42da-9bc2-72e0b925b579 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.314143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcbd9e-56b5-47db-95c9-4f49fc016dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.314821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcbd9e-56b5-47db-95c9-4f49fc016dec admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.315104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcbd9e-56b5-47db-95c9-4f49fc016dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.315375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcbd9e-56b5-47db-95c9-4f49fc016dec admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.316019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcbd9e-56b5-47db-95c9-4f49fc016dec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.317925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcbd9e-56b5-47db-95c9-4f49fc016dec admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.341071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1289/2607] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:01 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:42:01.345137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e49e69-31cb-4f58-afc8-5e7a7f5bda09 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.355341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e49e69-31cb-4f58-afc8-5e7a7f5bda09 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.360330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72491a1-daeb-4e18-956c-c3b0a9051ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.360942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72491a1-daeb-4e18-956c-c3b0a9051ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.361199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72491a1-daeb-4e18-956c-c3b0a9051ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.362020 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72491a1-daeb-4e18-956c-c3b0a9051ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.362665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72491a1-daeb-4e18-956c-c3b0a9051ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.364293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72491a1-daeb-4e18-956c-c3b0a9051ed6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.374902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1319/2608] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:01.381062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcbd9e-56b5-47db-95c9-4f49fc016dec admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.396621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcbd9e-56b5-47db-95c9-4f49fc016dec admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.402629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e3f23-57bc-4115-ad7e-4e8eefe5256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.403263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e3f23-57bc-4115-ad7e-4e8eefe5256e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.403567 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e3f23-57bc-4115-ad7e-4e8eefe5256e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.403839 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e3f23-57bc-4115-ad7e-4e8eefe5256e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.404475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e3f23-57bc-4115-ad7e-4e8eefe5256e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.406081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e3f23-57bc-4115-ad7e-4e8eefe5256e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.416685 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1290/2609] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:01.420844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72491a1-daeb-4e18-956c-c3b0a9051ed6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.431181 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72491a1-daeb-4e18-956c-c3b0a9051ed6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.436148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ca6a9-b1e6-4e47-84ad-47fc70a93df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.436774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca6a9-b1e6-4e47-84ad-47fc70a93df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.437053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca6a9-b1e6-4e47-84ad-47fc70a93df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.437323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca6a9-b1e6-4e47-84ad-47fc70a93df9 admin admin] PATH_INFO: `/v3/projects/da7e142d5e754fab941ce1a3779d5e53/users/82182874acd44f82924bf80733ee3b37/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.446116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:01.454083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ca6a9-b1e6-4e47-84ad-47fc70a93df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7e142d5e754fab941ce1a3779d5e53, user_id=82182874acd44f82924bf80733ee3b37, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.457709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ca6a9-b1e6-4e47-84ad-47fc70a93df9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.474670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1320/2610] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:01 2022] PUT /identity/v3/projects/da7e142d5e754fab941ce1a3779d5e53/users/82182874acd44f82924bf80733ee3b37/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:01.485557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c4270-5cb5-4e06-a5aa-5c173a62dca2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.485838 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c4270-5cb5-4e06-a5aa-5c173a62dca2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.486143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c4270-5cb5-4e06-a5aa-5c173a62dca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.515622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2f7c4270-5cb5-4e06-a5aa-5c173a62dca2 None None] MFA Rules not processed for user `82182874acd44f82924bf80733ee3b37`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:01.516832 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2f7c4270-5cb5-4e06-a5aa-5c173a62dca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:01.589180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1291/2611] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:01.593917 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ca6a9-b1e6-4e47-84ad-47fc70a93df9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.666804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ca6a9-b1e6-4e47-84ad-47fc70a93df9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.672367 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7134a-409b-4303-909b-bf61d01a4660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.672982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7134a-409b-4303-909b-bf61d01a4660 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.673258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7134a-409b-4303-909b-bf61d01a4660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.673525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7134a-409b-4303-909b-bf61d01a4660 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.674155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7134a-409b-4303-909b-bf61d01a4660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.676140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7134a-409b-4303-909b-bf61d01a4660 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.708152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1321/2612] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:01 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:01.712575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c4270-5cb5-4e06-a5aa-5c173a62dca2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.723728 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c4270-5cb5-4e06-a5aa-5c173a62dca2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.758312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:01.783227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b78b5-8964-4812-86d0-b35c2f8c3de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.783828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b78b5-8964-4812-86d0-b35c2f8c3de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.784101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b78b5-8964-4812-86d0-b35c2f8c3de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.784373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b78b5-8964-4812-86d0-b35c2f8c3de0 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.784998 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b78b5-8964-4812-86d0-b35c2f8c3de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.786961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b78b5-8964-4812-86d0-b35c2f8c3de0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.810887 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1292/2613] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:01 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:01.814945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7134a-409b-4303-909b-bf61d01a4660 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.825298 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7134a-409b-4303-909b-bf61d01a4660 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.830056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f4312-52f0-4b3a-b6ae-5d0bae3e5034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.830675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4312-52f0-4b3a-b6ae-5d0bae3e5034 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.830949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4312-52f0-4b3a-b6ae-5d0bae3e5034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.831220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f4312-52f0-4b3a-b6ae-5d0bae3e5034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.831861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f4312-52f0-4b3a-b6ae-5d0bae3e5034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.833521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f4312-52f0-4b3a-b6ae-5d0bae3e5034 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.844553 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1322/2614] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:01.848779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b78b5-8964-4812-86d0-b35c2f8c3de0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.859582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b78b5-8964-4812-86d0-b35c2f8c3de0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.865336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f36d2-d915-4de5-a6f1-8370e2afa9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.865996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f36d2-d915-4de5-a6f1-8370e2afa9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.866301 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f36d2-d915-4de5-a6f1-8370e2afa9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.866579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f36d2-d915-4de5-a6f1-8370e2afa9fe admin admin] PATH_INFO: `/v3/projects/bccda7239ddc405fb82b7c91c51ce59f/users/7d3c7476f894405990c568ee97006e80/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.882702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f36d2-d915-4de5-a6f1-8370e2afa9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccda7239ddc405fb82b7c91c51ce59f, user_id=7d3c7476f894405990c568ee97006e80, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.885418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f36d2-d915-4de5-a6f1-8370e2afa9fe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.899811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1293/2615] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:01 2022] PUT /identity/v3/projects/bccda7239ddc405fb82b7c91c51ce59f/users/7d3c7476f894405990c568ee97006e80/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:01.903864 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f4312-52f0-4b3a-b6ae-5d0bae3e5034 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:01.914222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f4312-52f0-4b3a-b6ae-5d0bae3e5034 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:01.980127 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822d1f4-3d1e-45da-af12-b841ac4277e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:01.980767 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d1f4-3d1e-45da-af12-b841ac4277e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:01.981025 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d1f4-3d1e-45da-af12-b841ac4277e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:01.981294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d1f4-3d1e-45da-af12-b841ac4277e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:01.981933 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822d1f4-3d1e-45da-af12-b841ac4277e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:01.983968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822d1f4-3d1e-45da-af12-b841ac4277e6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:01.994060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1323/2616] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:01.998423 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f36d2-d915-4de5-a6f1-8370e2afa9fe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:02.009342 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f36d2-d915-4de5-a6f1-8370e2afa9fe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:02.014850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:02.015497 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:02.015772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:02.016032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] PATH_INFO: `/v3/projects/bccda7239ddc405fb82b7c91c51ce59f/users/7d3c7476f894405990c568ee97006e80/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:02.017992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccda7239ddc405fb82b7c91c51ce59f, user_id=7d3c7476f894405990c568ee97006e80, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:02.020645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:02.035373 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1294/2617] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:01 2022] PUT /identity/v3/projects/bccda7239ddc405fb82b7c91c51ce59f/users/7d3c7476f894405990c568ee97006e80/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:02.046990 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875064-2da7-4895-a391-b81f35718567 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:02.047280 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875064-2da7-4895-a391-b81f35718567 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:02.047582 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875064-2da7-4895-a391-b81f35718567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:02.077209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-db875064-2da7-4895-a391-b81f35718567 None None] MFA Rules not processed for user `7d3c7476f894405990c568ee97006e80`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:02.078446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-db875064-2da7-4895-a391-b81f35718567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:02.124040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:02.150795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1324/2618] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:02.162833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:02.164868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:02.229326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb957-ad49-4e75-a427-d1b1a8b5f8cd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:02.233280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154631-b07c-400d-9ae9-8db8c0b149e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:02.234270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154631-b07c-400d-9ae9-8db8c0b149e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:02.234549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154631-b07c-400d-9ae9-8db8c0b149e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:02.234867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154631-b07c-400d-9ae9-8db8c0b149e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:02.244639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70154631-b07c-400d-9ae9-8db8c0b149e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:02.246076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70154631-b07c-400d-9ae9-8db8c0b149e7 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:02.255863 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1295/2619] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:02.327243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db875064-2da7-4895-a391-b81f35718567 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:02.412580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db875064-2da7-4895-a391-b81f35718567 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:02.418680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:02.419469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:02.419840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:02.420242 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] PATH_INFO: `/v3/users/82182874acd44f82924bf80733ee3b37` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:02.421502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82182874acd44f82924bf80733ee3b37)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:02.423583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:02.486523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82182874acd44f82924bf80733ee3b37'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:02.496111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82182874acd44f82924bf80733ee3b37'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:02.496772 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82182874acd44f82924bf80733ee3b37'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:02.514829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82182874acd44f82924bf80733ee3b37'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:02.515667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82182874acd44f82924bf80733ee3b37'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:02.528601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1325/2620] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:02 2022] DELETE /identity/v3/users/82182874acd44f82924bf80733ee3b37 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:02.533114 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41011589-2805-4ff7-ae77-5855651a5a87 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:02.533398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41011589-2805-4ff7-ae77-5855651a5a87 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:02.533667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41011589-2805-4ff7-ae77-5855651a5a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:02.553184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-41011589-2805-4ff7-ae77-5855651a5a87 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:02.555006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41011589-2805-4ff7-ae77-5855651a5a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:02.616093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1296/2621] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:42:02.627165 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:02.629502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:02.690040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555e725-523a-44fc-85d9-aa2a14ecbac6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:02.693744 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872982f-02e7-495b-89d7-0cc3f2533014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:02.694361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872982f-02e7-495b-89d7-0cc3f2533014 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:02.694633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872982f-02e7-495b-89d7-0cc3f2533014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:02.694922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872982f-02e7-495b-89d7-0cc3f2533014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:02.704309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872982f-02e7-495b-89d7-0cc3f2533014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:02.705739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872982f-02e7-495b-89d7-0cc3f2533014 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:02.715214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1326/2622] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:02.984454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41011589-2805-4ff7-ae77-5855651a5a87 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:02.986130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41011589-2805-4ff7-ae77-5855651a5a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:03.001251 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41011589-2805-4ff7-ae77-5855651a5a87 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:03.006158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:03.006827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:03.007100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:03.007369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] PATH_INFO: `/v3/projects/da7e142d5e754fab941ce1a3779d5e53` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:03.008508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7e142d5e754fab941ce1a3779d5e53)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:03.010495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:03.045393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:03.063136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7e142d5e754fab941ce1a3779d5e53', 'request_id': 'req-f30612f1-d722-41bd-adba-1d2f7ade7b46'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.071749 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7e142d5e754fab941ce1a3779d5e53', 'request_id': 'req-f30612f1-d722-41bd-adba-1d2f7ade7b46'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.081889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7e142d5e754fab941ce1a3779d5e53', 'request_id': 'req-f30612f1-d722-41bd-adba-1d2f7ade7b46'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.092555 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] The token cache is being invalidate because project da7e142d5e754fab941ce1a3779d5e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:03.092833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.093459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.095018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1297/2623] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:02 2022] DELETE /identity/v3/projects/da7e142d5e754fab941ce1a3779d5e53 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:03.099041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872982f-02e7-495b-89d7-0cc3f2533014 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:03.105113 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:03.109392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872982f-02e7-495b-89d7-0cc3f2533014 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:03.174499 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:03.175326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:03.175680 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:03.175998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] PATH_INFO: `/v3/users/7d3c7476f894405990c568ee97006e80` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:03.177804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3c7476f894405990c568ee97006e80)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:03.180831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:03.230522 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3c7476f894405990c568ee97006e80'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.238819 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3c7476f894405990c568ee97006e80'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.239455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3c7476f894405990c568ee97006e80'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.251969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3c7476f894405990c568ee97006e80'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.252607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-542e9939-1766-4946-aa07-a204327ccb0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3c7476f894405990c568ee97006e80'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.265779 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1327/2624] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:03 2022] DELETE /identity/v3/users/7d3c7476f894405990c568ee97006e80 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:03.545593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:03.547119 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:03.563356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30612f1-d722-41bd-adba-1d2f7ade7b46 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:03.624746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:03.625418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:03.625695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:03.625969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] PATH_INFO: `/v3/projects/bccda7239ddc405fb82b7c91c51ce59f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:03.627121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccda7239ddc405fb82b7c91c51ce59f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:03.629140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:03.681490 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccda7239ddc405fb82b7c91c51ce59f', 'request_id': 'req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.692051 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccda7239ddc405fb82b7c91c51ce59f', 'request_id': 'req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.701707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccda7239ddc405fb82b7c91c51ce59f', 'request_id': 'req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.712462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] The token cache is being invalidate because project bccda7239ddc405fb82b7c91c51ce59f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:03.712743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.713486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9cb64bd-cd89-4726-9eb7-5bf08ae786b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:03.715044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1298/2625] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:03 2022] DELETE /identity/v3/projects/bccda7239ddc405fb82b7c91c51ce59f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:03.719955 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d58c-e345-4cd6-a054-7964d36949b5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:03.720236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d58c-e345-4cd6-a054-7964d36949b5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:03.720501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78d58c-e345-4cd6-a054-7964d36949b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:03.740968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cb78d58c-e345-4cd6-a054-7964d36949b5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:03.742156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cb78d58c-e345-4cd6-a054-7964d36949b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:03.801453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1328/2626] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:03.809783 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361948c-d017-4d41-897f-cf9f1dab815e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:03.810087 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361948c-d017-4d41-897f-cf9f1dab815e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:03.810369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361948c-d017-4d41-897f-cf9f1dab815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:03.830869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b361948c-d017-4d41-897f-cf9f1dab815e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:03.832070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b361948c-d017-4d41-897f-cf9f1dab815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:03.841424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1299/2627] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:03.848283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78d58c-e345-4cd6-a054-7964d36949b5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:03.864432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78d58c-e345-4cd6-a054-7964d36949b5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:03.872044 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:03.872653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:03.872930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:03.873199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:03.873853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:42:03.874153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:03.875793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:03.887802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1329/2628] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:42:03.892212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361948c-d017-4d41-897f-cf9f1dab815e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:03.902015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361948c-d017-4d41-897f-cf9f1dab815e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:03.907818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78aeb-1a0a-4685-8fd3-f73cfd67f50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:03.908411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78aeb-1a0a-4685-8fd3-f73cfd67f50e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:03.908684 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78aeb-1a0a-4685-8fd3-f73cfd67f50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:03.908948 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc78aeb-1a0a-4685-8fd3-f73cfd67f50e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:03.909591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc78aeb-1a0a-4685-8fd3-f73cfd67f50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:03.911543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc78aeb-1a0a-4685-8fd3-f73cfd67f50e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:03.942978 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1300/2629] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:03 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:03.949082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:03.966750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efefc8-55ae-4e5b-9a24-aea9d160822a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:03.974915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:04.029222 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49bfe-d277-4e77-af54-8fb470674a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.029832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49bfe-d277-4e77-af54-8fb470674a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.030106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49bfe-d277-4e77-af54-8fb470674a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.030424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49bfe-d277-4e77-af54-8fb470674a14 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.031082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49bfe-d277-4e77-af54-8fb470674a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.033014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49bfe-d277-4e77-af54-8fb470674a14 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.056106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1330/2630] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:03 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:04.060439 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78aeb-1a0a-4685-8fd3-f73cfd67f50e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.071455 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc78aeb-1a0a-4685-8fd3-f73cfd67f50e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.077459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336e88e-535e-4cff-91ce-84ebe181cafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.078067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336e88e-535e-4cff-91ce-84ebe181cafb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.078343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336e88e-535e-4cff-91ce-84ebe181cafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.078606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336e88e-535e-4cff-91ce-84ebe181cafb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.079299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336e88e-535e-4cff-91ce-84ebe181cafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.080957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336e88e-535e-4cff-91ce-84ebe181cafb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.094761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1301/2631] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:04.096159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49bfe-d277-4e77-af54-8fb470674a14 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.106493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49bfe-d277-4e77-af54-8fb470674a14 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.111597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb5d32-d453-472f-ba1f-a388ef93998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.112194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5d32-d453-472f-ba1f-a388ef93998b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.112469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5d32-d453-472f-ba1f-a388ef93998b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.112736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5d32-d453-472f-ba1f-a388ef93998b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.113396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb5d32-d453-472f-ba1f-a388ef93998b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.115070 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb5d32-d453-472f-ba1f-a388ef93998b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.125913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1331/2632] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:04.130317 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336e88e-535e-4cff-91ce-84ebe181cafb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.140681 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336e88e-535e-4cff-91ce-84ebe181cafb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.145640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b432d9-71f8-4abd-b567-2097e22aa08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.146288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b432d9-71f8-4abd-b567-2097e22aa08d admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.146562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b432d9-71f8-4abd-b567-2097e22aa08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.146867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b432d9-71f8-4abd-b567-2097e22aa08d admin admin] PATH_INFO: `/v3/projects/e06dd7d23a0a437f9f34ee7d2fb3b3dd/users/2712b012704d471891f46372b4ad9cf9/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.149151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:04.165011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b432d9-71f8-4abd-b567-2097e22aa08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06dd7d23a0a437f9f34ee7d2fb3b3dd, user_id=2712b012704d471891f46372b4ad9cf9, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.167768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b432d9-71f8-4abd-b567-2097e22aa08d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.182617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1302/2633] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:04 2022] PUT /identity/v3/projects/e06dd7d23a0a437f9f34ee7d2fb3b3dd/users/2712b012704d471891f46372b4ad9cf9/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:04.194296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c8c8d-587b-4f90-ac88-3d971d80f132 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.194511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c8c8d-587b-4f90-ac88-3d971d80f132 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.194818 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c8c8d-587b-4f90-ac88-3d971d80f132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.224204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-645c8c8d-587b-4f90-ac88-3d971d80f132 None None] MFA Rules not processed for user `2712b012704d471891f46372b4ad9cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:04.225434 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-645c8c8d-587b-4f90-ac88-3d971d80f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:04.297024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1332/2634] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:04.305244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b432d9-71f8-4abd-b567-2097e22aa08d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.397023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b432d9-71f8-4abd-b567-2097e22aa08d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.402202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e0a81-0872-4f0d-bfe7-2c40261c41c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.402883 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0a81-0872-4f0d-bfe7-2c40261c41c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.403162 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0a81-0872-4f0d-bfe7-2c40261c41c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.403429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0a81-0872-4f0d-bfe7-2c40261c41c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.404056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e0a81-0872-4f0d-bfe7-2c40261c41c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.406860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e0a81-0872-4f0d-bfe7-2c40261c41c4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.439341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1303/2635] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:04 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:42:04.443781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c8c8d-587b-4f90-ac88-3d971d80f132 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.455649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c8c8d-587b-4f90-ac88-3d971d80f132 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.513824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee2cd6-ecc5-4690-bd5a-2c0cc75507e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.514443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2cd6-ecc5-4690-bd5a-2c0cc75507e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.514746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2cd6-ecc5-4690-bd5a-2c0cc75507e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.515022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2cd6-ecc5-4690-bd5a-2c0cc75507e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.515648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee2cd6-ecc5-4690-bd5a-2c0cc75507e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.517583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee2cd6-ecc5-4690-bd5a-2c0cc75507e4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.546951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1333/2636] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:04 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:04.553319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e0a81-0872-4f0d-bfe7-2c40261c41c4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.565786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e0a81-0872-4f0d-bfe7-2c40261c41c4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.571622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a83796-b65b-4594-a1b4-a8ca4337574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.572218 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a83796-b65b-4594-a1b4-a8ca4337574a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.572539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a83796-b65b-4594-a1b4-a8ca4337574a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.572995 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a83796-b65b-4594-a1b4-a8ca4337574a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.573442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a83796-b65b-4594-a1b4-a8ca4337574a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.575080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a83796-b65b-4594-a1b4-a8ca4337574a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.586414 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1304/2637] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:04.592285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee2cd6-ecc5-4690-bd5a-2c0cc75507e4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.603209 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee2cd6-ecc5-4690-bd5a-2c0cc75507e4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.609210 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa8aac-c4fc-49be-9b4c-78a62e841745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.609812 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa8aac-c4fc-49be-9b4c-78a62e841745 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.610085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa8aac-c4fc-49be-9b4c-78a62e841745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.610348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa8aac-c4fc-49be-9b4c-78a62e841745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.611009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa8aac-c4fc-49be-9b4c-78a62e841745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.612616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa8aac-c4fc-49be-9b4c-78a62e841745 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.615108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:04.627262 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1334/2638] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:04.632803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a83796-b65b-4594-a1b4-a8ca4337574a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.644131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a83796-b65b-4594-a1b4-a8ca4337574a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.649646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cfc52-5b59-48a0-8d0d-4df52aa510ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.650294 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfc52-5b59-48a0-8d0d-4df52aa510ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.650910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfc52-5b59-48a0-8d0d-4df52aa510ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.651192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cfc52-5b59-48a0-8d0d-4df52aa510ad admin admin] PATH_INFO: `/v3/projects/7fa6113f6a2241adb71d9b2a6f2e39fe/users/660dddffc5ba4191bbf39ace62e1a7bc/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.665855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cfc52-5b59-48a0-8d0d-4df52aa510ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa6113f6a2241adb71d9b2a6f2e39fe, user_id=660dddffc5ba4191bbf39ace62e1a7bc, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.668569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cfc52-5b59-48a0-8d0d-4df52aa510ad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.683448 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1305/2639] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:04 2022] PUT /identity/v3/projects/7fa6113f6a2241adb71d9b2a6f2e39fe/users/660dddffc5ba4191bbf39ace62e1a7bc/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:04.694137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727132-98d0-4d20-8cdc-ca4dc514b62b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.694417 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727132-98d0-4d20-8cdc-ca4dc514b62b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.694689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727132-98d0-4d20-8cdc-ca4dc514b62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.725727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-86727132-98d0-4d20-8cdc-ca4dc514b62b None None] MFA Rules not processed for user `660dddffc5ba4191bbf39ace62e1a7bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:04.727005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-86727132-98d0-4d20-8cdc-ca4dc514b62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:04.796796 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1335/2640] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:42:04.801258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cfc52-5b59-48a0-8d0d-4df52aa510ad admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.868303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cfc52-5b59-48a0-8d0d-4df52aa510ad admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.873055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a19e70-491c-4127-a35e-060e8c24b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.873658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a19e70-491c-4127-a35e-060e8c24b86c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.873930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a19e70-491c-4127-a35e-060e8c24b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.874197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a19e70-491c-4127-a35e-060e8c24b86c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.874850 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a19e70-491c-4127-a35e-060e8c24b86c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.876758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a19e70-491c-4127-a35e-060e8c24b86c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:04.906128 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1306/2641] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:04 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:04.910419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86727132-98d0-4d20-8cdc-ca4dc514b62b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:04.924440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86727132-98d0-4d20-8cdc-ca4dc514b62b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:04.988860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325b3ee-e378-48a8-9f41-1ee9d565facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:04.989481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325b3ee-e378-48a8-9f41-1ee9d565facf admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:04.989757 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325b3ee-e378-48a8-9f41-1ee9d565facf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:04.990024 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325b3ee-e378-48a8-9f41-1ee9d565facf admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:04.990661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325b3ee-e378-48a8-9f41-1ee9d565facf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:04.992659 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325b3ee-e378-48a8-9f41-1ee9d565facf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:05.017061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1336/2642] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:04 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:42:05.018516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a19e70-491c-4127-a35e-060e8c24b86c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:05.029137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a19e70-491c-4127-a35e-060e8c24b86c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:05.033933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc88033-a8f5-4f47-8cfb-80a01fc725f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:05.034509 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88033-a8f5-4f47-8cfb-80a01fc725f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:05.034881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88033-a8f5-4f47-8cfb-80a01fc725f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:05.035333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88033-a8f5-4f47-8cfb-80a01fc725f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:05.036045 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc88033-a8f5-4f47-8cfb-80a01fc725f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:05.037672 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc88033-a8f5-4f47-8cfb-80a01fc725f8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:05.178953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1307/2643] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:05.183624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325b3ee-e378-48a8-9f41-1ee9d565facf admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:05.194095 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325b3ee-e378-48a8-9f41-1ee9d565facf admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:05.199408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26413fc7-2ea0-4665-a5f4-d827dfb954f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:05.200068 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26413fc7-2ea0-4665-a5f4-d827dfb954f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:05.200463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26413fc7-2ea0-4665-a5f4-d827dfb954f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:05.200746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26413fc7-2ea0-4665-a5f4-d827dfb954f3 admin admin] PATH_INFO: `/v3/projects/b41768e18650455e8aaa30164e5176c2/users/9ddb6f71883b40539e92c9a0ed3d40d2/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:05.215931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26413fc7-2ea0-4665-a5f4-d827dfb954f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41768e18650455e8aaa30164e5176c2, user_id=9ddb6f71883b40539e92c9a0ed3d40d2, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:05.218590 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26413fc7-2ea0-4665-a5f4-d827dfb954f3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:05.233067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1337/2644] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:05 2022] PUT /identity/v3/projects/b41768e18650455e8aaa30164e5176c2/users/9ddb6f71883b40539e92c9a0ed3d40d2/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:05.237360 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc88033-a8f5-4f47-8cfb-80a01fc725f8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:05.247554 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc88033-a8f5-4f47-8cfb-80a01fc725f8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:05.251220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:05.308704 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237da2c-fe66-4d86-a697-0e00f7f942e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:05.309313 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237da2c-fe66-4d86-a697-0e00f7f942e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:05.309584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237da2c-fe66-4d86-a697-0e00f7f942e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:05.309846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237da2c-fe66-4d86-a697-0e00f7f942e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:05.310507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237da2c-fe66-4d86-a697-0e00f7f942e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:05.312129 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237da2c-fe66-4d86-a697-0e00f7f942e0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:05.322280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1308/2645] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:05.326528 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26413fc7-2ea0-4665-a5f4-d827dfb954f3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:05.337358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26413fc7-2ea0-4665-a5f4-d827dfb954f3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:05.342624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:05.343335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:05.343617 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:05.343887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] PATH_INFO: `/v3/projects/b41768e18650455e8aaa30164e5176c2/users/9ddb6f71883b40539e92c9a0ed3d40d2/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:05.345929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41768e18650455e8aaa30164e5176c2, user_id=9ddb6f71883b40539e92c9a0ed3d40d2, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:05.348658 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:05.360321 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1338/2646] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:05 2022] PUT /identity/v3/projects/b41768e18650455e8aaa30164e5176c2/users/9ddb6f71883b40539e92c9a0ed3d40d2/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:05.371059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4390d0b-9ea4-4e3c-a52f-af4122f57bab None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:05.371341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4390d0b-9ea4-4e3c-a52f-af4122f57bab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:05.371606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4390d0b-9ea4-4e3c-a52f-af4122f57bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:05.399453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d4390d0b-9ea4-4e3c-a52f-af4122f57bab None None] MFA Rules not processed for user `9ddb6f71883b40539e92c9a0ed3d40d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:05.400752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d4390d0b-9ea4-4e3c-a52f-af4122f57bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:05.471952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1309/2647] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:05.483139 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:05.485112 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:05.531010 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:05.546011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404827a-34ae-45d8-b792-a993b06d029d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:05.549869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93599b1-9c02-48f1-829f-1b3eb2df6c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:05.550479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93599b1-9c02-48f1-829f-1b3eb2df6c44 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:05.550747 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93599b1-9c02-48f1-829f-1b3eb2df6c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:05.551051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93599b1-9c02-48f1-829f-1b3eb2df6c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:05.560469 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93599b1-9c02-48f1-829f-1b3eb2df6c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:05.561909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93599b1-9c02-48f1-829f-1b3eb2df6c44 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:05.571970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1339/2648] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:06.123658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4390d0b-9ea4-4e3c-a52f-af4122f57bab None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:06.125174 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d4390d0b-9ea4-4e3c-a52f-af4122f57bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:06.140915 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4390d0b-9ea4-4e3c-a52f-af4122f57bab None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:06.145154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc6b9e-5c0e-44f5-a3de-960173005698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:06.145926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc6b9e-5c0e-44f5-a3de-960173005698 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:06.146298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc6b9e-5c0e-44f5-a3de-960173005698 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:06.146662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc6b9e-5c0e-44f5-a3de-960173005698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:06.219803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc6b9e-5c0e-44f5-a3de-960173005698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:06.221936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc6b9e-5c0e-44f5-a3de-960173005698 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:06.231500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1310/2649] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:10.367488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93599b1-9c02-48f1-829f-1b3eb2df6c44 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:10.377134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93599b1-9c02-48f1-829f-1b3eb2df6c44 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:10.381132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876f736-521a-49ac-9181-4106a96449a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:10.381739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876f736-521a-49ac-9181-4106a96449a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:10.382012 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876f736-521a-49ac-9181-4106a96449a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:10.382322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876f736-521a-49ac-9181-4106a96449a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:10.442058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876f736-521a-49ac-9181-4106a96449a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:10.443535 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876f736-521a-49ac-9181-4106a96449a4 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:10.452682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1340/2650] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:12.648386 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc6b9e-5c0e-44f5-a3de-960173005698 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:12.679024 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:12.714539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc6b9e-5c0e-44f5-a3de-960173005698 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:12.721646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:12.722264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:12.722557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:12.722855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] PATH_INFO: `/v3/users/2712b012704d471891f46372b4ad9cf9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:12.723989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2712b012704d471891f46372b4ad9cf9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:12.725941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:12.782992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2712b012704d471891f46372b4ad9cf9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:12.792336 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2712b012704d471891f46372b4ad9cf9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:12.792970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2712b012704d471891f46372b4ad9cf9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:12.807354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2712b012704d471891f46372b4ad9cf9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:12.808190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2712b012704d471891f46372b4ad9cf9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:12.820666 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1311/2651] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:12 2022] DELETE /identity/v3/users/2712b012704d471891f46372b4ad9cf9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:12.825354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b85a7-aa76-43bd-b9ff-75558dfb70f1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:12.825855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b85a7-aa76-43bd-b9ff-75558dfb70f1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:12.825970 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b85a7-aa76-43bd-b9ff-75558dfb70f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:12.846048 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7a2b85a7-aa76-43bd-b9ff-75558dfb70f1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:12.847294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7a2b85a7-aa76-43bd-b9ff-75558dfb70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:12.910303 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1341/2652] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:42:12.920069 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:12.921553 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:12.981973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5208a9-687e-48cd-a5d1-99f3ca6fb2e2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:12.986885 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f0634-79e2-435a-93e0-60c6e8854ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:12.987518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f0634-79e2-435a-93e0-60c6e8854ecd service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:12.987792 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f0634-79e2-435a-93e0-60c6e8854ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:12.988058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f0634-79e2-435a-93e0-60c6e8854ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:12.998299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f0634-79e2-435a-93e0-60c6e8854ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:13.000130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f0634-79e2-435a-93e0-60c6e8854ecd service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:13.009519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1312/2653] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:13.318323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b85a7-aa76-43bd-b9ff-75558dfb70f1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:13.319831 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7a2b85a7-aa76-43bd-b9ff-75558dfb70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:13.333108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b85a7-aa76-43bd-b9ff-75558dfb70f1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:13.338164 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:13.338922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:13.339430 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:13.340510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] PATH_INFO: `/v3/projects/e06dd7d23a0a437f9f34ee7d2fb3b3dd` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:13.341897 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06dd7d23a0a437f9f34ee7d2fb3b3dd)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:13.344045 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:13.398952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06dd7d23a0a437f9f34ee7d2fb3b3dd', 'request_id': 'req-c40dd89c-a668-4917-8a32-3f8cd9393eaa'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.407993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06dd7d23a0a437f9f34ee7d2fb3b3dd', 'request_id': 'req-c40dd89c-a668-4917-8a32-3f8cd9393eaa'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.414821 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06dd7d23a0a437f9f34ee7d2fb3b3dd', 'request_id': 'req-c40dd89c-a668-4917-8a32-3f8cd9393eaa'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.426241 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] The token cache is being invalidate because project e06dd7d23a0a437f9f34ee7d2fb3b3dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:13.426525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.427220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.428773 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1342/2654] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:13 2022] DELETE /identity/v3/projects/e06dd7d23a0a437f9f34ee7d2fb3b3dd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:13.432982 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f0634-79e2-435a-93e0-60c6e8854ecd service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:13.442579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f0634-79e2-435a-93e0-60c6e8854ecd service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:13.488472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:13.504077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:13.504722 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:13.505019 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:13.505290 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] PATH_INFO: `/v3/users/660dddffc5ba4191bbf39ace62e1a7bc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:13.506412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] RBAC: Authorizing `identity:delete_user(user_id=660dddffc5ba4191bbf39ace62e1a7bc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:13.508398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:13.560111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '660dddffc5ba4191bbf39ace62e1a7bc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.569488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '660dddffc5ba4191bbf39ace62e1a7bc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.570200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '660dddffc5ba4191bbf39ace62e1a7bc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.582677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '660dddffc5ba4191bbf39ace62e1a7bc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.583367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '660dddffc5ba4191bbf39ace62e1a7bc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:13.595796 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1313/2655] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:13 2022] DELETE /identity/v3/users/660dddffc5ba4191bbf39ace62e1a7bc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:13.914540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:13.916140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:13.926009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:13.930792 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd89c-a668-4917-8a32-3f8cd9393eaa admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:13.999424 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:14.000061 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:14.000328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:14.000601 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] PATH_INFO: `/v3/projects/7fa6113f6a2241adb71d9b2a6f2e39fe` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:14.001759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fa6113f6a2241adb71d9b2a6f2e39fe)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:14.003750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:14.053916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fa6113f6a2241adb71d9b2a6f2e39fe', 'request_id': 'req-40f6ad7b-64c7-44c6-9770-652140f316a9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.062507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fa6113f6a2241adb71d9b2a6f2e39fe', 'request_id': 'req-40f6ad7b-64c7-44c6-9770-652140f316a9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.071325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fa6113f6a2241adb71d9b2a6f2e39fe', 'request_id': 'req-40f6ad7b-64c7-44c6-9770-652140f316a9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.082933 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] The token cache is being invalidate because project 7fa6113f6a2241adb71d9b2a6f2e39fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:14.083225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.083954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.085502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1343/2656] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:13 2022] DELETE /identity/v3/projects/7fa6113f6a2241adb71d9b2a6f2e39fe => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:14.089857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:14.091400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:14.105027 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a41c0-584b-40c7-8568-1fac174fc26e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:14.167664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:14.168300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:14.168571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:14.168857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] PATH_INFO: `/v3/users/9ddb6f71883b40539e92c9a0ed3d40d2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:14.170406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ddb6f71883b40539e92c9a0ed3d40d2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:14.175121 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:14.238635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ddb6f71883b40539e92c9a0ed3d40d2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.249432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ddb6f71883b40539e92c9a0ed3d40d2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.250058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ddb6f71883b40539e92c9a0ed3d40d2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.260884 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:14.262478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ddb6f71883b40539e92c9a0ed3d40d2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.263276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58fce765-ab0a-4cfa-a816-c3d7be50f0e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ddb6f71883b40539e92c9a0ed3d40d2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.276910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1314/2657] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:14 2022] DELETE /identity/v3/users/9ddb6f71883b40539e92c9a0ed3d40d2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:14.563979 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:14.565475 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:14.580189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ad7b-64c7-44c6-9770-652140f316a9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:14.641651 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:14.642297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:14.642576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:14.642878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] PATH_INFO: `/v3/projects/b41768e18650455e8aaa30164e5176c2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:14.644034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41768e18650455e8aaa30164e5176c2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:14.645953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:14.686566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:14.697230 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41768e18650455e8aaa30164e5176c2', 'request_id': 'req-7ae84bfd-4d8b-4a48-900b-4390b86aee30'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.705754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41768e18650455e8aaa30164e5176c2', 'request_id': 'req-7ae84bfd-4d8b-4a48-900b-4390b86aee30'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.712041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41768e18650455e8aaa30164e5176c2', 'request_id': 'req-7ae84bfd-4d8b-4a48-900b-4390b86aee30'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.722597 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] The token cache is being invalidate because project b41768e18650455e8aaa30164e5176c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:14.722913 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.723527 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7ae84bfd-4d8b-4a48-900b-4390b86aee30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:14.724988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1344/2658] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:14 2022] DELETE /identity/v3/projects/b41768e18650455e8aaa30164e5176c2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:14.733346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108279ea-6b00-4e1f-a2b0-d9fc84506475 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:14.733632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108279ea-6b00-4e1f-a2b0-d9fc84506475 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:14.733898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108279ea-6b00-4e1f-a2b0-d9fc84506475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:14.754179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-108279ea-6b00-4e1f-a2b0-d9fc84506475 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:14.755331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-108279ea-6b00-4e1f-a2b0-d9fc84506475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:14.819427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1315/2659] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:14.829437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48d1a-a276-4f60-9268-a753a5848ff5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:14.829717 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48d1a-a276-4f60-9268-a753a5848ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:14.829992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f48d1a-a276-4f60-9268-a753a5848ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:14.849904 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-53f48d1a-a276-4f60-9268-a753a5848ff5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:14.851138 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-53f48d1a-a276-4f60-9268-a753a5848ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:14.860039 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1345/2660] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:14.864453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108279ea-6b00-4e1f-a2b0-d9fc84506475 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:14.874911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108279ea-6b00-4e1f-a2b0-d9fc84506475 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:14.880823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:14.881427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:14.881700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:14.881963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:14.882611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:42:14.882944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:14.884592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:14.896740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1316/2661] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:42:14.901255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f48d1a-a276-4f60-9268-a753a5848ff5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:14.910832 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f48d1a-a276-4f60-9268-a753a5848ff5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:14.916281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f117680-bd99-4395-bd6d-03d7bdabb22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:14.916903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117680-bd99-4395-bd6d-03d7bdabb22d admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:14.917310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117680-bd99-4395-bd6d-03d7bdabb22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:14.917608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117680-bd99-4395-bd6d-03d7bdabb22d admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:14.918243 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f117680-bd99-4395-bd6d-03d7bdabb22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:14.920267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f117680-bd99-4395-bd6d-03d7bdabb22d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:14.952002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1346/2662] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:14 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:42:14.956650 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:14.967040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071721d-da82-4d44-ab21-1f7284243304 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:15.024571 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cffed-48e6-4e13-81c4-54a97d8d8332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:15.025098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cffed-48e6-4e13-81c4-54a97d8d8332 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:15.025397 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cffed-48e6-4e13-81c4-54a97d8d8332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:15.025682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cffed-48e6-4e13-81c4-54a97d8d8332 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:15.026318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696cffed-48e6-4e13-81c4-54a97d8d8332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:15.028341 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696cffed-48e6-4e13-81c4-54a97d8d8332 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:15.054097 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1317/2663] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:14 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:15.059815 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f117680-bd99-4395-bd6d-03d7bdabb22d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:15.071033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f117680-bd99-4395-bd6d-03d7bdabb22d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:15.075500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b3dd4-dd85-45df-9236-df93afb43663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:15.076114 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b3dd4-dd85-45df-9236-df93afb43663 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:15.076391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b3dd4-dd85-45df-9236-df93afb43663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:15.076666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b3dd4-dd85-45df-9236-df93afb43663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:15.077331 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b3dd4-dd85-45df-9236-df93afb43663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:15.078988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b3dd4-dd85-45df-9236-df93afb43663 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:15.089718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1347/2664] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:15.094254 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cffed-48e6-4e13-81c4-54a97d8d8332 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:15.104613 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cffed-48e6-4e13-81c4-54a97d8d8332 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:15.109914 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f13c61-48e4-4015-992d-aa36953cba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:15.110654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f13c61-48e4-4015-992d-aa36953cba80 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:15.110969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f13c61-48e4-4015-992d-aa36953cba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:15.111231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f13c61-48e4-4015-992d-aa36953cba80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:15.111864 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f13c61-48e4-4015-992d-aa36953cba80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:15.113477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f13c61-48e4-4015-992d-aa36953cba80 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:15.124875 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1318/2665] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:15.129358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b3dd4-dd85-45df-9236-df93afb43663 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:15.139777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b3dd4-dd85-45df-9236-df93afb43663 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:15.144850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:15.145506 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:15.145781 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:15.146051 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] PATH_INFO: `/v3/projects/69713aca5ac144f9bca8348430ab9c75/users/81e0524ede63477a9875487d155be25e/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:15.160787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69713aca5ac144f9bca8348430ab9c75, user_id=81e0524ede63477a9875487d155be25e, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:15.163498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:15.179080 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1348/2666] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:15 2022] PUT /identity/v3/projects/69713aca5ac144f9bca8348430ab9c75/users/81e0524ede63477a9875487d155be25e/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:42:15.184429 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc74af-b1de-4283-a79b-5e703c3688bf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:15.184711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc74af-b1de-4283-a79b-5e703c3688bf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:15.184981 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc74af-b1de-4283-a79b-5e703c3688bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:15.205292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a6fc74af-b1de-4283-a79b-5e703c3688bf None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:15.206466 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a6fc74af-b1de-4283-a79b-5e703c3688bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:15.253585 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:15.273077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1319/2667] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:15.288269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:15.291569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:15.359509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f38a7d-2b98-497d-9407-7567371a4ba3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:15.364131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2359-5837-4b4d-9287-2db90feeaf52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:15.364741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2359-5837-4b4d-9287-2db90feeaf52 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:15.365015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2359-5837-4b4d-9287-2db90feeaf52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:15.365281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2359-5837-4b4d-9287-2db90feeaf52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:15.376237 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a2359-5837-4b4d-9287-2db90feeaf52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:15.377682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a2359-5837-4b4d-9287-2db90feeaf52 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:15.387944 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1349/2668] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:19.513037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e87e5-0cc0-454a-a37a-90d85f46a3a0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:19.513537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e87e5-0cc0-454a-a37a-90d85f46a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:19.513634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e87e5-0cc0-454a-a37a-90d85f46a3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:19.550756 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8b5e87e5-0cc0-454a-a37a-90d85f46a3a0 None None] MFA Rules not processed for user `81e0524ede63477a9875487d155be25e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:19.552116 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8b5e87e5-0cc0-454a-a37a-90d85f46a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:19.623963 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1320/2669] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:19.628470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2359-5837-4b4d-9287-2db90feeaf52 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:19.639654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2359-5837-4b4d-9287-2db90feeaf52 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:19.644358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215582b7-0d07-47a5-a798-818ec4a0bb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:19.644983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215582b7-0d07-47a5-a798-818ec4a0bb3b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:19.645267 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215582b7-0d07-47a5-a798-818ec4a0bb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:19.645557 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215582b7-0d07-47a5-a798-818ec4a0bb3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:19.646217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215582b7-0d07-47a5-a798-818ec4a0bb3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:19.648207 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215582b7-0d07-47a5-a798-818ec4a0bb3b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:19.677647 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1350/2670] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:19 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:19.682911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e87e5-0cc0-454a-a37a-90d85f46a3a0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:19.694136 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e87e5-0cc0-454a-a37a-90d85f46a3a0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:19.754669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cf666-5472-4a89-b342-803679231c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:19.755460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cf666-5472-4a89-b342-803679231c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:19.755740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cf666-5472-4a89-b342-803679231c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:19.756010 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cf666-5472-4a89-b342-803679231c8d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:19.756763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cf666-5472-4a89-b342-803679231c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:19.758738 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cf666-5472-4a89-b342-803679231c8d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:19.784662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1321/2671] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:19 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:19.788696 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215582b7-0d07-47a5-a798-818ec4a0bb3b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:19.799932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215582b7-0d07-47a5-a798-818ec4a0bb3b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:19.805206 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058f9ea-558c-420f-94e1-0e1dcb94e9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:19.805869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f9ea-558c-420f-94e1-0e1dcb94e9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:19.806148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f9ea-558c-420f-94e1-0e1dcb94e9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:19.806416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058f9ea-558c-420f-94e1-0e1dcb94e9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:19.807104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058f9ea-558c-420f-94e1-0e1dcb94e9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:19.809033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058f9ea-558c-420f-94e1-0e1dcb94e9f3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:19.820891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1351/2672] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:42:19.825297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cf666-5472-4a89-b342-803679231c8d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:19.835729 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cf666-5472-4a89-b342-803679231c8d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:19.840526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eef988-e7bc-48d4-93da-b7019a83f32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:19.841184 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eef988-e7bc-48d4-93da-b7019a83f32e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:19.841462 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eef988-e7bc-48d4-93da-b7019a83f32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:19.841735 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eef988-e7bc-48d4-93da-b7019a83f32e admin admin] PATH_INFO: `/v3/projects/ad13a1bcbf964a5fb2eb7cc577f8a8e3/users/5ea488935db448738c23d196a1a3b6b3/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:19.858511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eef988-e7bc-48d4-93da-b7019a83f32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad13a1bcbf964a5fb2eb7cc577f8a8e3, user_id=5ea488935db448738c23d196a1a3b6b3, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:19.861583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eef988-e7bc-48d4-93da-b7019a83f32e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:19.878205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1322/2673] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:19 2022] PUT /identity/v3/projects/ad13a1bcbf964a5fb2eb7cc577f8a8e3/users/5ea488935db448738c23d196a1a3b6b3/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:19.882086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058f9ea-558c-420f-94e1-0e1dcb94e9f3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:19.893707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058f9ea-558c-420f-94e1-0e1dcb94e9f3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:19.925576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:19.955878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d7df6-cef7-47f7-8c0f-9d18d694a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:19.956500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d7df6-cef7-47f7-8c0f-9d18d694a5ed admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:19.956783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d7df6-cef7-47f7-8c0f-9d18d694a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:19.957054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d7df6-cef7-47f7-8c0f-9d18d694a5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:19.957703 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d7df6-cef7-47f7-8c0f-9d18d694a5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:19.959374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d7df6-cef7-47f7-8c0f-9d18d694a5ed admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:19.970479 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1352/2674] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:19.974925 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eef988-e7bc-48d4-93da-b7019a83f32e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:19.986548 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eef988-e7bc-48d4-93da-b7019a83f32e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:19.993023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:19.993683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:19.993960 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:19.994230 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] PATH_INFO: `/v3/projects/ad13a1bcbf964a5fb2eb7cc577f8a8e3/users/5ea488935db448738c23d196a1a3b6b3/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:19.996440 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad13a1bcbf964a5fb2eb7cc577f8a8e3, user_id=5ea488935db448738c23d196a1a3b6b3, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:19.999214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:20.010157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1323/2675] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:19 2022] PUT /identity/v3/projects/ad13a1bcbf964a5fb2eb7cc577f8a8e3/users/5ea488935db448738c23d196a1a3b6b3/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:25.058758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed86878-e9e3-4bd1-a7ca-3d57230f7725 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:25.059841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed86878-e9e3-4bd1-a7ca-3d57230f7725 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:25.060144 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed86878-e9e3-4bd1-a7ca-3d57230f7725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:25.088999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8ed86878-e9e3-4bd1-a7ca-3d57230f7725 None None] MFA Rules not processed for user `5ea488935db448738c23d196a1a3b6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:25.090211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8ed86878-e9e3-4bd1-a7ca-3d57230f7725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:25.162709 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1353/2676] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:25.177449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:25.179052 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:25.224564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:25.263955 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa17986-4cbc-42ee-8b22-093fc13a9f21 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:25.270857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ba928-7575-4620-914d-6c28c55f6b83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:25.271480 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ba928-7575-4620-914d-6c28c55f6b83 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:25.271926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ba928-7575-4620-914d-6c28c55f6b83 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:25.272200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ba928-7575-4620-914d-6c28c55f6b83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:25.340742 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ba928-7575-4620-914d-6c28c55f6b83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:25.342557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ba928-7575-4620-914d-6c28c55f6b83 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:25.352866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1324/2677] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:26.616887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9531-5901-4a53-9b55-fc2cf238160f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:26.617881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9531-5901-4a53-9b55-fc2cf238160f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:26.618341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9531-5901-4a53-9b55-fc2cf238160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:26.638959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-df6a9531-5901-4a53-9b55-fc2cf238160f None None] MFA Rules not processed for user `81e0524ede63477a9875487d155be25e`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:26.640192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-df6a9531-5901-4a53-9b55-fc2cf238160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:26.651116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1354/2678] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:27.375941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ba928-7575-4620-914d-6c28c55f6b83 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:27.431622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ba928-7575-4620-914d-6c28c55f6b83 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:27.435510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:27.436096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:27.436367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:27.436632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:27.438356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:27.505779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:27.507763 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:27.521299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1325/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:31.034017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9531-5901-4a53-9b55-fc2cf238160f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:31.036125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-df6a9531-5901-4a53-9b55-fc2cf238160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:31.058100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9531-5901-4a53-9b55-fc2cf238160f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:31.064149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:31.064931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.065442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.065922 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] PATH_INFO: `/v3/users/13ac62bdb9bc4c7f94dad08f810559e9` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.067337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ac62bdb9bc4c7f94dad08f810559e9)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:31.069774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:31.158350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ac62bdb9bc4c7f94dad08f810559e9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.167364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ac62bdb9bc4c7f94dad08f810559e9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.168030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ac62bdb9bc4c7f94dad08f810559e9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.185285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ac62bdb9bc4c7f94dad08f810559e9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.186106 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6e309c5e-18a9-476a-a2df-2cf343815a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ac62bdb9bc4c7f94dad08f810559e9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.198115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1355/2680] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:31 2022] DELETE /identity/v3/users/13ac62bdb9bc4c7f94dad08f810559e9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:31.383852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:31.394179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b0b5b-9438-47b9-9d86-637f4e792ea9 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:31.416738 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:31.456700 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:31.457330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.457606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.457874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] PATH_INFO: `/v3/projects/3bd54eb751f144b29b274e6a96dedcb2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.459155 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd54eb751f144b29b274e6a96dedcb2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:31.461246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:31.519288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd54eb751f144b29b274e6a96dedcb2', 'request_id': 'req-417055fd-f3fe-4f1e-9c12-04fc058a2543'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.525378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd54eb751f144b29b274e6a96dedcb2', 'request_id': 'req-417055fd-f3fe-4f1e-9c12-04fc058a2543'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.534584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd54eb751f144b29b274e6a96dedcb2', 'request_id': 'req-417055fd-f3fe-4f1e-9c12-04fc058a2543'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.545951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] The token cache is being invalidate because project 3bd54eb751f144b29b274e6a96dedcb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:31.546236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.546899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-417055fd-f3fe-4f1e-9c12-04fc058a2543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:31.548415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1326/2681] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:31 2022] DELETE /identity/v3/projects/3bd54eb751f144b29b274e6a96dedcb2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:31.554140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40c4e1-e1f5-4232-b321-5c1b52747eaa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.554423 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40c4e1-e1f5-4232-b321-5c1b52747eaa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.554692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40c4e1-e1f5-4232-b321-5c1b52747eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.576159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ed40c4e1-e1f5-4232-b321-5c1b52747eaa None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:31.577389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ed40c4e1-e1f5-4232-b321-5c1b52747eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:31.590146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:31.642264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1356/2682] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:31.650561 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e071de3-1ccb-448c-a6c3-9d9f86b5f224 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.650842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e071de3-1ccb-448c-a6c3-9d9f86b5f224 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.651137 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e071de3-1ccb-448c-a6c3-9d9f86b5f224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.672609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7e071de3-1ccb-448c-a6c3-9d9f86b5f224 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:31.673728 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7e071de3-1ccb-448c-a6c3-9d9f86b5f224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:31.683371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1327/2683] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:31.688276 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40c4e1-e1f5-4232-b321-5c1b52747eaa None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:31.698369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40c4e1-e1f5-4232-b321-5c1b52747eaa None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:31.703609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:31.704216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.704488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.704780 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.705462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:42:31.705766 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:31.707426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:31.720291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1357/2684] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:42:31.729326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e071de3-1ccb-448c-a6c3-9d9f86b5f224 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:31.741122 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e071de3-1ccb-448c-a6c3-9d9f86b5f224 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:31.747236 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde8ed-0235-42a7-8cf0-a00ef367f1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:31.747898 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde8ed-0235-42a7-8cf0-a00ef367f1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.748125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde8ed-0235-42a7-8cf0-a00ef367f1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.748402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde8ed-0235-42a7-8cf0-a00ef367f1a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.749049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cde8ed-0235-42a7-8cf0-a00ef367f1a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:31.751026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cde8ed-0235-42a7-8cf0-a00ef367f1a6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:31.786946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1328/2685] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:31 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:31.788433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:31.799777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad65db6-a35a-4744-af17-5f70264ad8da admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:31.862879 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83fc53-c66e-4ac3-8060-c87e30859a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:31.863507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fc53-c66e-4ac3-8060-c87e30859a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.863782 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fc53-c66e-4ac3-8060-c87e30859a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.864072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fc53-c66e-4ac3-8060-c87e30859a10 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.864753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83fc53-c66e-4ac3-8060-c87e30859a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:31.866678 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83fc53-c66e-4ac3-8060-c87e30859a10 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:31.890674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1358/2686] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:31 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:31.894902 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde8ed-0235-42a7-8cf0-a00ef367f1a6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:31.905813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde8ed-0235-42a7-8cf0-a00ef367f1a6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:31.910510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0efd75-47fe-4808-827c-4fa72941294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:31.911154 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0efd75-47fe-4808-827c-4fa72941294c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.911428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0efd75-47fe-4808-827c-4fa72941294c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.911692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0efd75-47fe-4808-827c-4fa72941294c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.912332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0efd75-47fe-4808-827c-4fa72941294c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:31.913938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0efd75-47fe-4808-827c-4fa72941294c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:31.924333 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1329/2687] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:31.928455 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83fc53-c66e-4ac3-8060-c87e30859a10 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:31.939740 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83fc53-c66e-4ac3-8060-c87e30859a10 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:31.945536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142ad75-cad0-47e8-ab75-a7776cc1ee8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:31.946135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142ad75-cad0-47e8-ab75-a7776cc1ee8d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.946418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142ad75-cad0-47e8-ab75-a7776cc1ee8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.946817 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142ad75-cad0-47e8-ab75-a7776cc1ee8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:31.947708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142ad75-cad0-47e8-ab75-a7776cc1ee8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:31.949356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142ad75-cad0-47e8-ab75-a7776cc1ee8d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:31.963926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1359/2688] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:31.971660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0efd75-47fe-4808-827c-4fa72941294c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:31.982880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0efd75-47fe-4808-827c-4fa72941294c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:31.989233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5bc88-24f3-44b9-bf1b-0ec3fa208d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:31.989892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5bc88-24f3-44b9-bf1b-0ec3fa208d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:31.990165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5bc88-24f3-44b9-bf1b-0ec3fa208d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:31.990475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5bc88-24f3-44b9-bf1b-0ec3fa208d7b admin admin] PATH_INFO: `/v3/projects/616ad85f2f4f4d77bef25a1ef39ce1b7/users/b68a2c9493f64612ae1b358ad594107d/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.004826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5bc88-24f3-44b9-bf1b-0ec3fa208d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=616ad85f2f4f4d77bef25a1ef39ce1b7, user_id=b68a2c9493f64612ae1b358ad594107d, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:32.007563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5bc88-24f3-44b9-bf1b-0ec3fa208d7b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:32.022651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1330/2689] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:31 2022] PUT /identity/v3/projects/616ad85f2f4f4d77bef25a1ef39ce1b7/users/b68a2c9493f64612ae1b358ad594107d/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:32.033128 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e652e-21b9-425c-b6ec-44cd14c50fd0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.033412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e652e-21b9-425c-b6ec-44cd14c50fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.033687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e652e-21b9-425c-b6ec-44cd14c50fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.065411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2b2e652e-21b9-425c-b6ec-44cd14c50fd0 None None] MFA Rules not processed for user `b68a2c9493f64612ae1b358ad594107d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:32.066632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2b2e652e-21b9-425c-b6ec-44cd14c50fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:32.137946 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1360/2690] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:32.142453 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5bc88-24f3-44b9-bf1b-0ec3fa208d7b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:32.198820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:32.213189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5bc88-24f3-44b9-bf1b-0ec3fa208d7b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:32.221765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c9cbc-710f-4bbe-8c8f-7df6e01be775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:32.222388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c9cbc-710f-4bbe-8c8f-7df6e01be775 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.222662 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c9cbc-710f-4bbe-8c8f-7df6e01be775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.222970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c9cbc-710f-4bbe-8c8f-7df6e01be775 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.223637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c9cbc-710f-4bbe-8c8f-7df6e01be775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:32.225569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c9cbc-710f-4bbe-8c8f-7df6e01be775 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:32.260367 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1331/2691] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:32 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:32.261618 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e652e-21b9-425c-b6ec-44cd14c50fd0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:32.274692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e652e-21b9-425c-b6ec-44cd14c50fd0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:32.285783 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:32.344124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d83507-17db-4e38-ac13-0534823490bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:32.344803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d83507-17db-4e38-ac13-0534823490bf admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.345083 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d83507-17db-4e38-ac13-0534823490bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.345354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d83507-17db-4e38-ac13-0534823490bf admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.345987 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d83507-17db-4e38-ac13-0534823490bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:32.347945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d83507-17db-4e38-ac13-0534823490bf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:32.373693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1361/2692] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:32 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:32.377636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c9cbc-710f-4bbe-8c8f-7df6e01be775 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:32.388377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c9cbc-710f-4bbe-8c8f-7df6e01be775 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:32.393706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f45895-b6a5-4c52-8f2f-8f70acde2aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:32.394486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45895-b6a5-4c52-8f2f-8f70acde2aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.394896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45895-b6a5-4c52-8f2f-8f70acde2aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.395091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45895-b6a5-4c52-8f2f-8f70acde2aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.398057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f45895-b6a5-4c52-8f2f-8f70acde2aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:32.398057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f45895-b6a5-4c52-8f2f-8f70acde2aa7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:32.410491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1332/2693] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:42:32.414796 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d83507-17db-4e38-ac13-0534823490bf admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:32.425132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d83507-17db-4e38-ac13-0534823490bf admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:32.430324 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7e515-f5cf-47f0-b572-68439bbd9724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:32.431094 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7e515-f5cf-47f0-b572-68439bbd9724 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.431989 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7e515-f5cf-47f0-b572-68439bbd9724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.432358 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7e515-f5cf-47f0-b572-68439bbd9724 admin admin] PATH_INFO: `/v3/projects/71d398c8bf0b4e02a8e3362b171d021e/users/2384cc4196ce4313b4c934ed0ccd9d20/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.449948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7e515-f5cf-47f0-b572-68439bbd9724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d398c8bf0b4e02a8e3362b171d021e, user_id=2384cc4196ce4313b4c934ed0ccd9d20, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:32.452736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7e515-f5cf-47f0-b572-68439bbd9724 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:32.468227 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1362/2694] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:32 2022] PUT /identity/v3/projects/71d398c8bf0b4e02a8e3362b171d021e/users/2384cc4196ce4313b4c934ed0ccd9d20/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:32.472202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f45895-b6a5-4c52-8f2f-8f70acde2aa7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:32.482502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f45895-b6a5-4c52-8f2f-8f70acde2aa7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:32.542115 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3cd42-f39e-41d3-933a-b25baa4d3ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:32.542755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3cd42-f39e-41d3-933a-b25baa4d3ead admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.543061 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3cd42-f39e-41d3-933a-b25baa4d3ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.543348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3cd42-f39e-41d3-933a-b25baa4d3ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.543985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3cd42-f39e-41d3-933a-b25baa4d3ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:32.545754 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3cd42-f39e-41d3-933a-b25baa4d3ead admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:32.556398 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1333/2695] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:32.560992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7e515-f5cf-47f0-b572-68439bbd9724 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:32.571572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7e515-f5cf-47f0-b572-68439bbd9724 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:32.577403 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:32.578057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.578333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.578592 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] PATH_INFO: `/v3/projects/71d398c8bf0b4e02a8e3362b171d021e/users/2384cc4196ce4313b4c934ed0ccd9d20/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.580771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d398c8bf0b4e02a8e3362b171d021e, user_id=2384cc4196ce4313b4c934ed0ccd9d20, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:32.583460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:32.594753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1363/2696] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:32 2022] PUT /identity/v3/projects/71d398c8bf0b4e02a8e3362b171d021e/users/2384cc4196ce4313b4c934ed0ccd9d20/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:32.606400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71566324-6198-40c3-aa17-9aa4fa490877 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.606682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71566324-6198-40c3-aa17-9aa4fa490877 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.606975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71566324-6198-40c3-aa17-9aa4fa490877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.636971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-71566324-6198-40c3-aa17-9aa4fa490877 None None] MFA Rules not processed for user `2384cc4196ce4313b4c934ed0ccd9d20`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:32.638334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-71566324-6198-40c3-aa17-9aa4fa490877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:32.710417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1334/2697] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:32.722786 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:32.724795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:32.784503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8ecb0-dfbd-4067-88fa-b731fc2274a8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:32.788503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81f06-a981-4f67-b795-335ae9090283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:32.789116 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81f06-a981-4f67-b795-335ae9090283 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:32.789395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81f06-a981-4f67-b795-335ae9090283 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:32.789663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81f06-a981-4f67-b795-335ae9090283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:32.800148 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81f06-a981-4f67-b795-335ae9090283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:32.801610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81f06-a981-4f67-b795-335ae9090283 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:32.811504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1364/2698] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:34.333278 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71566324-6198-40c3-aa17-9aa4fa490877 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:34.384100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:34.404653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71566324-6198-40c3-aa17-9aa4fa490877 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:34.409765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:34.410519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:34.410831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:34.411105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] PATH_INFO: `/v3/users/b68a2c9493f64612ae1b358ad594107d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:34.412307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68a2c9493f64612ae1b358ad594107d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:34.414237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:34.462457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68a2c9493f64612ae1b358ad594107d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:34.471153 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68a2c9493f64612ae1b358ad594107d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:34.471873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68a2c9493f64612ae1b358ad594107d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:34.486177 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68a2c9493f64612ae1b358ad594107d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:34.488385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68a2c9493f64612ae1b358ad594107d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:34.503573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1335/2699] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:34 2022] DELETE /identity/v3/users/b68a2c9493f64612ae1b358ad594107d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:34.508412 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15ad27-372e-4282-bbb5-033514939899 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:34.508948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15ad27-372e-4282-bbb5-033514939899 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:34.508948 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15ad27-372e-4282-bbb5-033514939899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:34.529294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bc15ad27-372e-4282-bbb5-033514939899 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:34.530855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bc15ad27-372e-4282-bbb5-033514939899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:34.595453 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1365/2700] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:34.605471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:34.607100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:34.674298 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685dfff-9527-4700-a60e-8043f517f146 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:34.681130 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ff0ee-82a6-43de-93ad-5b6ef744c094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:34.681953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff0ee-82a6-43de-93ad-5b6ef744c094 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:34.682368 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff0ee-82a6-43de-93ad-5b6ef744c094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:34.682671 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ff0ee-82a6-43de-93ad-5b6ef744c094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:34.693247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ff0ee-82a6-43de-93ad-5b6ef744c094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:34.694734 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ff0ee-82a6-43de-93ad-5b6ef744c094 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:34.704852 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1336/2701] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:34.983860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15ad27-372e-4282-bbb5-033514939899 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:34.985719 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bc15ad27-372e-4282-bbb5-033514939899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:35.007477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15ad27-372e-4282-bbb5-033514939899 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:35.013375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:35.014143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:35.014515 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:35.014914 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] PATH_INFO: `/v3/projects/616ad85f2f4f4d77bef25a1ef39ce1b7` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:35.016057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=616ad85f2f4f4d77bef25a1ef39ce1b7)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:35.017999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:35.020551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:35.074355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '616ad85f2f4f4d77bef25a1ef39ce1b7', 'request_id': 'req-f39d9fc5-8a89-4478-9e7e-e389204859e8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.083789 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '616ad85f2f4f4d77bef25a1ef39ce1b7', 'request_id': 'req-f39d9fc5-8a89-4478-9e7e-e389204859e8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.091074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '616ad85f2f4f4d77bef25a1ef39ce1b7', 'request_id': 'req-f39d9fc5-8a89-4478-9e7e-e389204859e8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.103607 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] The token cache is being invalidate because project 616ad85f2f4f4d77bef25a1ef39ce1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:35.103972 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.104969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.107130 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1366/2702] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:34 2022] DELETE /identity/v3/projects/616ad85f2f4f4d77bef25a1ef39ce1b7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:35.111547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ff0ee-82a6-43de-93ad-5b6ef744c094 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:35.121165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ff0ee-82a6-43de-93ad-5b6ef744c094 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:35.183127 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:35.183791 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:35.184068 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:35.184326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] PATH_INFO: `/v3/users/2384cc4196ce4313b4c934ed0ccd9d20` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:35.185637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2384cc4196ce4313b4c934ed0ccd9d20)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:35.187872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:35.194932 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:35.243006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2384cc4196ce4313b4c934ed0ccd9d20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.252933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2384cc4196ce4313b4c934ed0ccd9d20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.253592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2384cc4196ce4313b4c934ed0ccd9d20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.265602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2384cc4196ce4313b4c934ed0ccd9d20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.266334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e21e367b-bbd9-48ed-a295-c6b5d876b285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2384cc4196ce4313b4c934ed0ccd9d20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.281563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1337/2703] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:35 2022] DELETE /identity/v3/users/2384cc4196ce4313b4c934ed0ccd9d20 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:35.578398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:35.579962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:35.602286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d9fc5-8a89-4478-9e7e-e389204859e8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:35.669966 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:35.670516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:35.670825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:35.671097 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] PATH_INFO: `/v3/projects/71d398c8bf0b4e02a8e3362b171d021e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:35.672266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d398c8bf0b4e02a8e3362b171d021e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:35.674152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:35.726785 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d398c8bf0b4e02a8e3362b171d021e', 'request_id': 'req-a29ab50f-7f79-4d9e-9f93-940c47b620e9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.737356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d398c8bf0b4e02a8e3362b171d021e', 'request_id': 'req-a29ab50f-7f79-4d9e-9f93-940c47b620e9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.743971 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d398c8bf0b4e02a8e3362b171d021e', 'request_id': 'req-a29ab50f-7f79-4d9e-9f93-940c47b620e9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.758366 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] The token cache is being invalidate because project 71d398c8bf0b4e02a8e3362b171d021e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:35.758623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.759425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a29ab50f-7f79-4d9e-9f93-940c47b620e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:35.761337 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1367/2704] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:35 2022] DELETE /identity/v3/projects/71d398c8bf0b4e02a8e3362b171d021e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:35.766703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5ca61-1cdf-45b8-b76f-3105d0ba3071 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:35.767277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5ca61-1cdf-45b8-b76f-3105d0ba3071 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:35.767277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5ca61-1cdf-45b8-b76f-3105d0ba3071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:35.787782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0ed5ca61-1cdf-45b8-b76f-3105d0ba3071 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:35.789002 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ed5ca61-1cdf-45b8-b76f-3105d0ba3071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:35.850159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1338/2705] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:35.858534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ae8a-e1e2-47fd-91be-c412bdda4b5b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:35.858845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ae8a-e1e2-47fd-91be-c412bdda4b5b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:35.859118 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ae8a-e1e2-47fd-91be-c412bdda4b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:35.879473 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fd76ae8a-e1e2-47fd-91be-c412bdda4b5b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:35.880691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fd76ae8a-e1e2-47fd-91be-c412bdda4b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:35.888999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1368/2706] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:35.893434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5ca61-1cdf-45b8-b76f-3105d0ba3071 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:35.904332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5ca61-1cdf-45b8-b76f-3105d0ba3071 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:35.909483 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:35.910100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:35.910375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:35.910643 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:35.911412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:42:35.911692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:35.913291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:35.926654 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1339/2707] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:42:35.934163 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76ae8a-e1e2-47fd-91be-c412bdda4b5b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:35.944054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76ae8a-e1e2-47fd-91be-c412bdda4b5b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:35.949391 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466480c3-f97c-458e-8e86-62bfd4d91b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:35.950009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466480c3-f97c-458e-8e86-62bfd4d91b7b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:35.950394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466480c3-f97c-458e-8e86-62bfd4d91b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:35.950939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466480c3-f97c-458e-8e86-62bfd4d91b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:35.951823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466480c3-f97c-458e-8e86-62bfd4d91b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:35.954023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466480c3-f97c-458e-8e86-62bfd4d91b7b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:35.970452 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:35.985134 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1369/2708] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:35 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:35.989450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:35.999936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8369099-c97e-48ce-a8e3-5ef3ee38fb8f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.056979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbdc2f-cfe7-42d4-bafa-5174b3b2add9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.057589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbdc2f-cfe7-42d4-bafa-5174b3b2add9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.057835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbdc2f-cfe7-42d4-bafa-5174b3b2add9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.058134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbdc2f-cfe7-42d4-bafa-5174b3b2add9 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.058779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbdc2f-cfe7-42d4-bafa-5174b3b2add9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.060779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbdc2f-cfe7-42d4-bafa-5174b3b2add9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.084782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1340/2709] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:35 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:36.089179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466480c3-f97c-458e-8e86-62bfd4d91b7b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.100670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466480c3-f97c-458e-8e86-62bfd4d91b7b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.105861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f974f-572c-43b5-9810-57e1c204f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.106485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f974f-572c-43b5-9810-57e1c204f134 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.106840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f974f-572c-43b5-9810-57e1c204f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.107299 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f974f-572c-43b5-9810-57e1c204f134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.108274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f974f-572c-43b5-9810-57e1c204f134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.109623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f974f-572c-43b5-9810-57e1c204f134 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.120681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1370/2710] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:36.125034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbdc2f-cfe7-42d4-bafa-5174b3b2add9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.130424 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:36.135990 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbdc2f-cfe7-42d4-bafa-5174b3b2add9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.142388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a8d13-299d-44fc-b69b-6e124cf061fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.143841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8d13-299d-44fc-b69b-6e124cf061fd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.144197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8d13-299d-44fc-b69b-6e124cf061fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.144499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8d13-299d-44fc-b69b-6e124cf061fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.145201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a8d13-299d-44fc-b69b-6e124cf061fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.147192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a8d13-299d-44fc-b69b-6e124cf061fd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.159867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1341/2711] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:42:36.164378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f974f-572c-43b5-9810-57e1c204f134 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.176260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f974f-572c-43b5-9810-57e1c204f134 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.181838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ce7d3-fc39-4b3d-afa8-4434a0c1ac7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.182485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ce7d3-fc39-4b3d-afa8-4434a0c1ac7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.182770 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ce7d3-fc39-4b3d-afa8-4434a0c1ac7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.183052 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ce7d3-fc39-4b3d-afa8-4434a0c1ac7f admin admin] PATH_INFO: `/v3/projects/19d5041b17e54a06bc99f411466f7f2e/users/ce6d67870bd04077a9937d32c7459450/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.201059 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ce7d3-fc39-4b3d-afa8-4434a0c1ac7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d5041b17e54a06bc99f411466f7f2e, user_id=ce6d67870bd04077a9937d32c7459450, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.204100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ce7d3-fc39-4b3d-afa8-4434a0c1ac7f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.220546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1371/2712] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:36 2022] PUT /identity/v3/projects/19d5041b17e54a06bc99f411466f7f2e/users/ce6d67870bd04077a9937d32c7459450/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:36.232340 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16bfff-bebf-4bd3-962b-afd43fec5745 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.232623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16bfff-bebf-4bd3-962b-afd43fec5745 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.232924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16bfff-bebf-4bd3-962b-afd43fec5745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.262223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fb16bfff-bebf-4bd3-962b-afd43fec5745 None None] MFA Rules not processed for user `ce6d67870bd04077a9937d32c7459450`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:36.263720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb16bfff-bebf-4bd3-962b-afd43fec5745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:36.331488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1342/2713] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:36.336327 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ce7d3-fc39-4b3d-afa8-4434a0c1ac7f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.410758 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ce7d3-fc39-4b3d-afa8-4434a0c1ac7f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.415959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f63c41-4e8d-4955-8b7a-8366b2fc280e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.416568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f63c41-4e8d-4955-8b7a-8366b2fc280e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.416847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f63c41-4e8d-4955-8b7a-8366b2fc280e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.417120 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f63c41-4e8d-4955-8b7a-8366b2fc280e admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.417753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f63c41-4e8d-4955-8b7a-8366b2fc280e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.419961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f63c41-4e8d-4955-8b7a-8366b2fc280e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.449722 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1372/2714] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:36 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:36.454109 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16bfff-bebf-4bd3-962b-afd43fec5745 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.465170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16bfff-bebf-4bd3-962b-afd43fec5745 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.524050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73772599-7501-4cee-80fa-b9920543a599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.524682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772599-7501-4cee-80fa-b9920543a599 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.524946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772599-7501-4cee-80fa-b9920543a599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.525214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772599-7501-4cee-80fa-b9920543a599 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.525815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73772599-7501-4cee-80fa-b9920543a599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.527770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73772599-7501-4cee-80fa-b9920543a599 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.550945 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1343/2715] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:36 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:36.555032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f63c41-4e8d-4955-8b7a-8366b2fc280e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.566670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f63c41-4e8d-4955-8b7a-8366b2fc280e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.572551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0d92e-934f-4e6c-90e3-78aad34abb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.573161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0d92e-934f-4e6c-90e3-78aad34abb32 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.573435 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0d92e-934f-4e6c-90e3-78aad34abb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.573701 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0d92e-934f-4e6c-90e3-78aad34abb32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.574364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0d92e-934f-4e6c-90e3-78aad34abb32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.576085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0d92e-934f-4e6c-90e3-78aad34abb32 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.587355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1373/2716] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:36.591766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73772599-7501-4cee-80fa-b9920543a599 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.607583 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73772599-7501-4cee-80fa-b9920543a599 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.613436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a478c-d5ca-4c32-b685-c1b9e555d8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.614393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a478c-d5ca-4c32-b685-c1b9e555d8d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.614393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a478c-d5ca-4c32-b685-c1b9e555d8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.614679 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a478c-d5ca-4c32-b685-c1b9e555d8d1 admin admin] PATH_INFO: `/v3/projects/0bad30de971744b8b83d76d472c87a9f/users/d8735f3b9ae949d0b9c61edb3e6802d1/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.633893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a478c-d5ca-4c32-b685-c1b9e555d8d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bad30de971744b8b83d76d472c87a9f, user_id=d8735f3b9ae949d0b9c61edb3e6802d1, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.636828 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a478c-d5ca-4c32-b685-c1b9e555d8d1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.650871 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1344/2717] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:36 2022] PUT /identity/v3/projects/0bad30de971744b8b83d76d472c87a9f/users/d8735f3b9ae949d0b9c61edb3e6802d1/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:36.654999 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0d92e-934f-4e6c-90e3-78aad34abb32 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.665353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0d92e-934f-4e6c-90e3-78aad34abb32 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.722285 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa58d10-e105-4c5d-ad5c-3ac2822a4b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.722860 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa58d10-e105-4c5d-ad5c-3ac2822a4b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.723159 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa58d10-e105-4c5d-ad5c-3ac2822a4b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.723427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa58d10-e105-4c5d-ad5c-3ac2822a4b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.724067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa58d10-e105-4c5d-ad5c-3ac2822a4b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.725681 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa58d10-e105-4c5d-ad5c-3ac2822a4b83 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.735952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1374/2718] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:36.741661 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a478c-d5ca-4c32-b685-c1b9e555d8d1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.754709 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a478c-d5ca-4c32-b685-c1b9e555d8d1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.759867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.760515 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.760818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.761072 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] PATH_INFO: `/v3/projects/0bad30de971744b8b83d76d472c87a9f/users/d8735f3b9ae949d0b9c61edb3e6802d1/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.763829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bad30de971744b8b83d76d472c87a9f, user_id=d8735f3b9ae949d0b9c61edb3e6802d1, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.766500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:36.777606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1345/2719] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:36 2022] PUT /identity/v3/projects/0bad30de971744b8b83d76d472c87a9f/users/d8735f3b9ae949d0b9c61edb3e6802d1/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:36.787935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7c95-9484-4d12-b8eb-e7dec23cd8f6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.788218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7c95-9484-4d12-b8eb-e7dec23cd8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.788487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7c95-9484-4d12-b8eb-e7dec23cd8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.818311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ea4b7c95-9484-4d12-b8eb-e7dec23cd8f6 None None] MFA Rules not processed for user `d8735f3b9ae949d0b9c61edb3e6802d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:36.819552 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ea4b7c95-9484-4d12-b8eb-e7dec23cd8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:36.844351 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:36.897049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1375/2720] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:36.908533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:36.911568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:36.956437 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:36.981818 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17a8cc-4a1d-4b55-9508-0a2788d34fb6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:36.986434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df020a-4910-412b-ba17-cdb072685219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:36.987076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df020a-4910-412b-ba17-cdb072685219 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:36.987351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df020a-4910-412b-ba17-cdb072685219 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:36.987620 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df020a-4910-412b-ba17-cdb072685219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:36.997574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df020a-4910-412b-ba17-cdb072685219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:36.999063 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df020a-4910-412b-ba17-cdb072685219 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:37.008471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1346/2721] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:37.632344 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b7c95-9484-4d12-b8eb-e7dec23cd8f6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:37.698829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b7c95-9484-4d12-b8eb-e7dec23cd8f6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:37.703320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:37.703953 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:37.704226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:37.704492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] PATH_INFO: `/v3/users/ce6d67870bd04077a9937d32c7459450` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:37.705588 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6d67870bd04077a9937d32c7459450)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:37.707605 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:37.754938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6d67870bd04077a9937d32c7459450'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:37.763877 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6d67870bd04077a9937d32c7459450'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:37.764556 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6d67870bd04077a9937d32c7459450'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:37.776375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6d67870bd04077a9937d32c7459450'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:37.776983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6d67870bd04077a9937d32c7459450'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:37.788902 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1376/2722] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:37 2022] DELETE /identity/v3/users/ce6d67870bd04077a9937d32c7459450 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:37.793517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120c14-dc3c-48e5-b340-22f4abf5d2b5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:37.794025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120c14-dc3c-48e5-b340-22f4abf5d2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:37.794144 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120c14-dc3c-48e5-b340-22f4abf5d2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:37.813018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cb120c14-dc3c-48e5-b340-22f4abf5d2b5 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:37.814189 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cb120c14-dc3c-48e5-b340-22f4abf5d2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:37.876046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1347/2723] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:37.888843 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:37.890318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:37.928194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:37.954504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbd9fa-b946-4f98-be96-b018fff7e5e6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:37.958631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc85451-6de3-4cd5-9440-837c55c3fa33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:37.959269 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85451-6de3-4cd5-9440-837c55c3fa33 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:37.959540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85451-6de3-4cd5-9440-837c55c3fa33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:37.959806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85451-6de3-4cd5-9440-837c55c3fa33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:37.969244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc85451-6de3-4cd5-9440-837c55c3fa33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:37.970676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc85451-6de3-4cd5-9440-837c55c3fa33 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:37.980133 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1377/2724] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:42:38.252081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120c14-dc3c-48e5-b340-22f4abf5d2b5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:38.253591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cb120c14-dc3c-48e5-b340-22f4abf5d2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:38.265911 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120c14-dc3c-48e5-b340-22f4abf5d2b5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:38.270582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:38.271255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:38.271550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:38.271840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] PATH_INFO: `/v3/projects/19d5041b17e54a06bc99f411466f7f2e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:38.273065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d5041b17e54a06bc99f411466f7f2e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:38.275037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:38.322217 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d5041b17e54a06bc99f411466f7f2e', 'request_id': 'req-6548a4ec-bd38-45c0-9072-8e858089d693'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.328421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d5041b17e54a06bc99f411466f7f2e', 'request_id': 'req-6548a4ec-bd38-45c0-9072-8e858089d693'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.337229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d5041b17e54a06bc99f411466f7f2e', 'request_id': 'req-6548a4ec-bd38-45c0-9072-8e858089d693'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.348918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] The token cache is being invalidate because project 19d5041b17e54a06bc99f411466f7f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:38.349198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.349812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.351326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1348/2725] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:38 2022] DELETE /identity/v3/projects/19d5041b17e54a06bc99f411466f7f2e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:38.355400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc85451-6de3-4cd5-9440-837c55c3fa33 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:38.366030 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc85451-6de3-4cd5-9440-837c55c3fa33 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:38.433562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:38.434244 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:38.434540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:38.434858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] PATH_INFO: `/v3/users/d8735f3b9ae949d0b9c61edb3e6802d1` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:38.436078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8735f3b9ae949d0b9c61edb3e6802d1)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:38.438075 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:38.487996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8735f3b9ae949d0b9c61edb3e6802d1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.497216 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8735f3b9ae949d0b9c61edb3e6802d1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.498282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8735f3b9ae949d0b9c61edb3e6802d1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.511153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8735f3b9ae949d0b9c61edb3e6802d1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.511799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c7669e76-c907-4485-9146-27745d070c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8735f3b9ae949d0b9c61edb3e6802d1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.524145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1378/2726] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:38 2022] DELETE /identity/v3/users/d8735f3b9ae949d0b9c61edb3e6802d1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:38.807385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:38.808953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:38.822411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548a4ec-bd38-45c0-9072-8e858089d693 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:38.879745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:38.880435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:38.880655 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:38.880920 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] PATH_INFO: `/v3/projects/0bad30de971744b8b83d76d472c87a9f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:38.882030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bad30de971744b8b83d76d472c87a9f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:38.883967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:38.885388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:38.933007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bad30de971744b8b83d76d472c87a9f', 'request_id': 'req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.939111 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bad30de971744b8b83d76d472c87a9f', 'request_id': 'req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.948023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bad30de971744b8b83d76d472c87a9f', 'request_id': 'req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.958745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] The token cache is being invalidate because project 0bad30de971744b8b83d76d472c87a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:38.959040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.959695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fc07e898-6ffc-40ad-8bdb-83fd27ca2a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:38.961192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1349/2727] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:38 2022] DELETE /identity/v3/projects/0bad30de971744b8b83d76d472c87a9f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:38.966257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050b100-5e1e-4ac2-9e94-9a7291714693 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:38.966741 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050b100-5e1e-4ac2-9e94-9a7291714693 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:38.966840 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050b100-5e1e-4ac2-9e94-9a7291714693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:38.987440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e050b100-5e1e-4ac2-9e94-9a7291714693 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:38.988660 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e050b100-5e1e-4ac2-9e94-9a7291714693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:39.043005 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:39.060993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1379/2728] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:39.068918 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812f919-0874-4f41-9472-b359db2d37fe None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.069208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812f919-0874-4f41-9472-b359db2d37fe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.069506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812f919-0874-4f41-9472-b359db2d37fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.090842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c812f919-0874-4f41-9472-b359db2d37fe None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:39.092670 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c812f919-0874-4f41-9472-b359db2d37fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:39.101004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1350/2729] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:39.105433 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050b100-5e1e-4ac2-9e94-9a7291714693 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.116599 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050b100-5e1e-4ac2-9e94-9a7291714693 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.121692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.122310 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.123029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.123309 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.123985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:42:39.124289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.125907 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.138875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1380/2730] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:42:39.143637 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812f919-0874-4f41-9472-b359db2d37fe None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.153238 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812f919-0874-4f41-9472-b359db2d37fe None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.159186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e9372-5469-48e2-bfca-7540bef52d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.159766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e9372-5469-48e2-bfca-7540bef52d39 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.160062 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e9372-5469-48e2-bfca-7540bef52d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.160330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e9372-5469-48e2-bfca-7540bef52d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.160968 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e9372-5469-48e2-bfca-7540bef52d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.162901 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e9372-5469-48e2-bfca-7540bef52d39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.191016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1351/2731] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:39 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:42:39.195293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.205870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6978c50-8341-494c-b5e0-a190a25913d5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.267802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269d06d-88f8-4307-a331-5d5fe9603a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.268468 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269d06d-88f8-4307-a331-5d5fe9603a27 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.268727 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269d06d-88f8-4307-a331-5d5fe9603a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.268998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269d06d-88f8-4307-a331-5d5fe9603a27 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.269623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269d06d-88f8-4307-a331-5d5fe9603a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.271587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269d06d-88f8-4307-a331-5d5fe9603a27 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.294903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1381/2732] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:39 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:39.299235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e9372-5469-48e2-bfca-7540bef52d39 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.310016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e9372-5469-48e2-bfca-7540bef52d39 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.317175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd66c3-0c28-453b-8b2b-df5c7bfd6a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.318025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd66c3-0c28-453b-8b2b-df5c7bfd6a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.318436 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd66c3-0c28-453b-8b2b-df5c7bfd6a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.318767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd66c3-0c28-453b-8b2b-df5c7bfd6a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.319689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd66c3-0c28-453b-8b2b-df5c7bfd6a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.322188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd66c3-0c28-453b-8b2b-df5c7bfd6a62 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.334201 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1352/2733] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:39.338248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269d06d-88f8-4307-a331-5d5fe9603a27 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.348688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269d06d-88f8-4307-a331-5d5fe9603a27 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.353504 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5a40-4cc6-41cf-a669-141cd8a98fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.354096 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5a40-4cc6-41cf-a669-141cd8a98fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.354376 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5a40-4cc6-41cf-a669-141cd8a98fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.354656 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5a40-4cc6-41cf-a669-141cd8a98fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.355347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5a40-4cc6-41cf-a669-141cd8a98fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.357027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5a40-4cc6-41cf-a669-141cd8a98fc6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.368186 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1382/2734] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:39.372635 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd66c3-0c28-453b-8b2b-df5c7bfd6a62 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.384140 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd66c3-0c28-453b-8b2b-df5c7bfd6a62 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.390816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96638e06-0582-4b6e-b6ac-5cd9eace4653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.391404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96638e06-0582-4b6e-b6ac-5cd9eace4653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.391669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96638e06-0582-4b6e-b6ac-5cd9eace4653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.391936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96638e06-0582-4b6e-b6ac-5cd9eace4653 admin admin] PATH_INFO: `/v3/projects/4c95567878bc427785cbb3e0afcbbda9/users/3d58c3b6663247ca92d4d2561f865c8c/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.411054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96638e06-0582-4b6e-b6ac-5cd9eace4653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c95567878bc427785cbb3e0afcbbda9, user_id=3d58c3b6663247ca92d4d2561f865c8c, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.413717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96638e06-0582-4b6e-b6ac-5cd9eace4653 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.433034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1353/2735] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:39 2022] PUT /identity/v3/projects/4c95567878bc427785cbb3e0afcbbda9/users/3d58c3b6663247ca92d4d2561f865c8c/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:42:39.448673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19f583-21f9-47be-a8e1-589aa250e1c3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.448959 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19f583-21f9-47be-a8e1-589aa250e1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.449231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19f583-21f9-47be-a8e1-589aa250e1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.479834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9d19f583-21f9-47be-a8e1-589aa250e1c3 None None] MFA Rules not processed for user `3d58c3b6663247ca92d4d2561f865c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:39.481042 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9d19f583-21f9-47be-a8e1-589aa250e1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:39.547540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1383/2736] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:39.552425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96638e06-0582-4b6e-b6ac-5cd9eace4653 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.621787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96638e06-0582-4b6e-b6ac-5cd9eace4653 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.627133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ef1a9-d4ad-4b64-a444-2718c0678dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.627724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ef1a9-d4ad-4b64-a444-2718c0678dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.627999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ef1a9-d4ad-4b64-a444-2718c0678dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.628346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ef1a9-d4ad-4b64-a444-2718c0678dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.628937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ef1a9-d4ad-4b64-a444-2718c0678dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.630889 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ef1a9-d4ad-4b64-a444-2718c0678dc4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.659931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1354/2737] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:39 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:39.664131 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19f583-21f9-47be-a8e1-589aa250e1c3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.675084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19f583-21f9-47be-a8e1-589aa250e1c3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.720871 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:39.731870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37aff8-7dfc-470e-b17c-3aa710477ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.732490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37aff8-7dfc-470e-b17c-3aa710477ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.732761 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37aff8-7dfc-470e-b17c-3aa710477ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.733029 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37aff8-7dfc-470e-b17c-3aa710477ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.733666 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37aff8-7dfc-470e-b17c-3aa710477ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.735988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37aff8-7dfc-470e-b17c-3aa710477ba2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.762361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1384/2738] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:39 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:39.763459 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ef1a9-d4ad-4b64-a444-2718c0678dc4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.774263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ef1a9-d4ad-4b64-a444-2718c0678dc4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.779710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c7123-af8a-41e6-acab-5ef69543eb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.780300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c7123-af8a-41e6-acab-5ef69543eb19 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.780575 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c7123-af8a-41e6-acab-5ef69543eb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.780849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c7123-af8a-41e6-acab-5ef69543eb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.781522 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c7123-af8a-41e6-acab-5ef69543eb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.783231 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c7123-af8a-41e6-acab-5ef69543eb19 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.788642 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:39.803691 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1355/2739] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:39.808013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37aff8-7dfc-470e-b17c-3aa710477ba2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.818442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37aff8-7dfc-470e-b17c-3aa710477ba2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.823396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d0973-7a38-4c87-b1ee-710e426d13c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.824049 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d0973-7a38-4c87-b1ee-710e426d13c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.824329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d0973-7a38-4c87-b1ee-710e426d13c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.824603 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d0973-7a38-4c87-b1ee-710e426d13c5 admin admin] PATH_INFO: `/v3/projects/e45340007df3428b9c174dae195104d8/users/3f9db1df3f884e6b87e7ad98defe9d6a/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.841197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d0973-7a38-4c87-b1ee-710e426d13c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45340007df3428b9c174dae195104d8, user_id=3f9db1df3f884e6b87e7ad98defe9d6a, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.844027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d0973-7a38-4c87-b1ee-710e426d13c5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.859349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1385/2740] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:39 2022] PUT /identity/v3/projects/e45340007df3428b9c174dae195104d8/users/3f9db1df3f884e6b87e7ad98defe9d6a/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:39.864054 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c7123-af8a-41e6-acab-5ef69543eb19 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.874245 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c7123-af8a-41e6-acab-5ef69543eb19 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.930800 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a340b-3c95-4a79-9608-d784c5a155ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.931357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a340b-3c95-4a79-9608-d784c5a155ef admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.931626 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a340b-3c95-4a79-9608-d784c5a155ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.931890 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a340b-3c95-4a79-9608-d784c5a155ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.932648 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a340b-3c95-4a79-9608-d784c5a155ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.934269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a340b-3c95-4a79-9608-d784c5a155ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.944841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1356/2741] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:39.949547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d0973-7a38-4c87-b1ee-710e426d13c5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:39.960238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d0973-7a38-4c87-b1ee-710e426d13c5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:39.964975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:39.965624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.965919 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.966257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] PATH_INFO: `/v3/projects/e45340007df3428b9c174dae195104d8/users/3f9db1df3f884e6b87e7ad98defe9d6a/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:39.968311 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45340007df3428b9c174dae195104d8, user_id=3f9db1df3f884e6b87e7ad98defe9d6a, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:39.970975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:39.980911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1386/2742] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:39 2022] PUT /identity/v3/projects/e45340007df3428b9c174dae195104d8/users/3f9db1df3f884e6b87e7ad98defe9d6a/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:39.991751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee61c-52df-4ea6-bf7d-22dfc57a7940 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:39.992037 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee61c-52df-4ea6-bf7d-22dfc57a7940 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:39.992303 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee61c-52df-4ea6-bf7d-22dfc57a7940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:40.021606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c3eee61c-52df-4ea6-bf7d-22dfc57a7940 None None] MFA Rules not processed for user `3f9db1df3f884e6b87e7ad98defe9d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:40.022826 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c3eee61c-52df-4ea6-bf7d-22dfc57a7940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:40.094695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1357/2743] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:40.105646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:40.107931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:40.167406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8b750-5a4d-4302-97db-e8ce443df311 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:40.171629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236839f-b22a-448d-85fe-69f2116c3d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:40.172217 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236839f-b22a-448d-85fe-69f2116c3d47 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:40.172494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236839f-b22a-448d-85fe-69f2116c3d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:40.172754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236839f-b22a-448d-85fe-69f2116c3d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:40.182122 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236839f-b22a-448d-85fe-69f2116c3d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:40.183525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236839f-b22a-448d-85fe-69f2116c3d47 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:40.192934 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1387/2744] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:40.594232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eee61c-52df-4ea6-bf7d-22dfc57a7940 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:40.656677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eee61c-52df-4ea6-bf7d-22dfc57a7940 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:40.661297 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:40.661916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:40.662200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:40.662451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] PATH_INFO: `/v3/users/3d58c3b6663247ca92d4d2561f865c8c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:40.663623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d58c3b6663247ca92d4d2561f865c8c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:40.665533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:40.688093 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:40.721185 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d58c3b6663247ca92d4d2561f865c8c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:40.730627 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d58c3b6663247ca92d4d2561f865c8c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:40.731261 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d58c3b6663247ca92d4d2561f865c8c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:40.743410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d58c3b6663247ca92d4d2561f865c8c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:40.744081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d58c3b6663247ca92d4d2561f865c8c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:40.756450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1358/2745] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:40 2022] DELETE /identity/v3/users/3d58c3b6663247ca92d4d2561f865c8c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:40.761254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6abfd-6caf-42fa-b9d4-3f11f85a792a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:40.761547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6abfd-6caf-42fa-b9d4-3f11f85a792a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:40.761853 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6abfd-6caf-42fa-b9d4-3f11f85a792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:40.781827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-87c6abfd-6caf-42fa-b9d4-3f11f85a792a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:40.783204 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-87c6abfd-6caf-42fa-b9d4-3f11f85a792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:40.846328 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1388/2746] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:40.856724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:40.858422 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:40.916539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f106b9-4bec-49fa-9c30-c6ce047a9bd4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:40.921170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fad4d7-8eec-4cca-9340-d297bb66117d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:40.922009 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fad4d7-8eec-4cca-9340-d297bb66117d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:40.922369 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fad4d7-8eec-4cca-9340-d297bb66117d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:40.922766 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fad4d7-8eec-4cca-9340-d297bb66117d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:40.932359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fad4d7-8eec-4cca-9340-d297bb66117d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:40.933867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fad4d7-8eec-4cca-9340-d297bb66117d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:40.943531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1359/2747] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:41.226952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6abfd-6caf-42fa-b9d4-3f11f85a792a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:41.228460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-87c6abfd-6caf-42fa-b9d4-3f11f85a792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:41.241021 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6abfd-6caf-42fa-b9d4-3f11f85a792a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:41.248155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:41.248798 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:41.249071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:41.249343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] PATH_INFO: `/v3/projects/4c95567878bc427785cbb3e0afcbbda9` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:41.250476 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c95567878bc427785cbb3e0afcbbda9)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:41.252442 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:41.301000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:41.307969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c95567878bc427785cbb3e0afcbbda9', 'request_id': 'req-7d3ee975-463d-4749-b66d-38acc5599787'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.316602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c95567878bc427785cbb3e0afcbbda9', 'request_id': 'req-7d3ee975-463d-4749-b66d-38acc5599787'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.322931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c95567878bc427785cbb3e0afcbbda9', 'request_id': 'req-7d3ee975-463d-4749-b66d-38acc5599787'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.333043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] The token cache is being invalidate because project 4c95567878bc427785cbb3e0afcbbda9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:41.333323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.333981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.335490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1389/2748] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:41 2022] DELETE /identity/v3/projects/4c95567878bc427785cbb3e0afcbbda9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:41.340178 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fad4d7-8eec-4cca-9340-d297bb66117d service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:41.350471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fad4d7-8eec-4cca-9340-d297bb66117d service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:41.411801 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:41.412451 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:41.412724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:41.412988 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] PATH_INFO: `/v3/users/3f9db1df3f884e6b87e7ad98defe9d6a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:41.414088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9db1df3f884e6b87e7ad98defe9d6a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:41.416021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:41.461747 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:41.463326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9db1df3f884e6b87e7ad98defe9d6a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.473388 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9db1df3f884e6b87e7ad98defe9d6a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.474324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9db1df3f884e6b87e7ad98defe9d6a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.494984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9db1df3f884e6b87e7ad98defe9d6a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.495755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c4d542b-c7ed-4c71-b079-9617f92a462d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9db1df3f884e6b87e7ad98defe9d6a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.508992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1360/2749] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:41 2022] DELETE /identity/v3/users/3f9db1df3f884e6b87e7ad98defe9d6a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:41.791099 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:41.792650 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:41.806622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ee975-463d-4749-b66d-38acc5599787 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:41.866606 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:41.867353 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:41.867593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:41.867869 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] PATH_INFO: `/v3/projects/e45340007df3428b9c174dae195104d8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:41.869927 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45340007df3428b9c174dae195104d8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:41.870958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:41.926221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45340007df3428b9c174dae195104d8', 'request_id': 'req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.935090 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45340007df3428b9c174dae195104d8', 'request_id': 'req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.943712 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45340007df3428b9c174dae195104d8', 'request_id': 'req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.956349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] The token cache is being invalidate because project e45340007df3428b9c174dae195104d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:41.956687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.957405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5c67caaa-3d1d-49fd-97af-f6e8745b07a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:41.959472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1390/2750] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:41 2022] DELETE /identity/v3/projects/e45340007df3428b9c174dae195104d8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:41.964718 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4c4f-b4bb-4434-b150-75cf6c923705 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:41.965214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4c4f-b4bb-4434-b150-75cf6c923705 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:41.965214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4c4f-b4bb-4434-b150-75cf6c923705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:41.986385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b0ea4c4f-b4bb-4434-b150-75cf6c923705 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:41.987673 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b0ea4c4f-b4bb-4434-b150-75cf6c923705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:42.047817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1361/2751] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:42:42.056623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc0c5-f84f-48e8-a2c3-4299ab632e54 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.056977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc0c5-f84f-48e8-a2c3-4299ab632e54 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.057378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc0c5-f84f-48e8-a2c3-4299ab632e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.081372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c4abc0c5-f84f-48e8-a2c3-4299ab632e54 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:42.082578 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c4abc0c5-f84f-48e8-a2c3-4299ab632e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:42.090357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1391/2752] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:42.094685 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea4c4f-b4bb-4434-b150-75cf6c923705 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:42.104142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea4c4f-b4bb-4434-b150-75cf6c923705 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:42.108603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:42.109209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.109476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.109739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.110812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:42:42.111104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:42.112657 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:42.125135 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1362/2753] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:42:42.129885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc0c5-f84f-48e8-a2c3-4299ab632e54 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:42.139936 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc0c5-f84f-48e8-a2c3-4299ab632e54 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:42.145689 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859133b-5e04-4de2-ad60-fbc0546b2f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:42.146399 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859133b-5e04-4de2-ad60-fbc0546b2f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.146682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859133b-5e04-4de2-ad60-fbc0546b2f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.146984 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859133b-5e04-4de2-ad60-fbc0546b2f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.147620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859133b-5e04-4de2-ad60-fbc0546b2f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:42.149537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859133b-5e04-4de2-ad60-fbc0546b2f69 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:42.185107 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1392/2754] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:42 2022] POST /identity/v3/projects => generated 361 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:42.189399 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:42.200611 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e78589-3a0e-40ae-99bb-ecf6a0db4c6f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:42.269798 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265f86b-3e18-4517-ae79-06c124e0f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:42.270667 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265f86b-3e18-4517-ae79-06c124e0f088 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.271081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265f86b-3e18-4517-ae79-06c124e0f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.271598 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265f86b-3e18-4517-ae79-06c124e0f088 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.272208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265f86b-3e18-4517-ae79-06c124e0f088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:42.274232 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265f86b-3e18-4517-ae79-06c124e0f088 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:42.301482 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1363/2755] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:42 2022] POST /identity/v3/users => generated 281 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:42.303231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859133b-5e04-4de2-ad60-fbc0546b2f69 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:42.316214 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859133b-5e04-4de2-ad60-fbc0546b2f69 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:42.326602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6c518-23e0-4dff-8082-d5fa1e2fd46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:42.327281 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c518-23e0-4dff-8082-d5fa1e2fd46f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.327580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c518-23e0-4dff-8082-d5fa1e2fd46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.327824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c518-23e0-4dff-8082-d5fa1e2fd46f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.328490 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c518-23e0-4dff-8082-d5fa1e2fd46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:42.330098 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c518-23e0-4dff-8082-d5fa1e2fd46f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:42.344396 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1393/2756] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:42.349396 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265f86b-3e18-4517-ae79-06c124e0f088 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:42.360371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265f86b-3e18-4517-ae79-06c124e0f088 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:42.365629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46c864-d5b7-49d3-8783-bd915c4abca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:42.366469 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46c864-d5b7-49d3-8783-bd915c4abca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.366842 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46c864-d5b7-49d3-8783-bd915c4abca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.367203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46c864-d5b7-49d3-8783-bd915c4abca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.367970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46c864-d5b7-49d3-8783-bd915c4abca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:42.369640 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46c864-d5b7-49d3-8783-bd915c4abca2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:42.387055 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1364/2757] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:42:42.388498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6c518-23e0-4dff-8082-d5fa1e2fd46f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:42.400228 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6c518-23e0-4dff-8082-d5fa1e2fd46f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:42.405294 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:42.405983 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.406306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.406575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] PATH_INFO: `/v3/projects/695b71b9060343dab99d3388ce6794e3/users/9d495a4106fa4a9eab8a2a8198ef9849/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.422704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=695b71b9060343dab99d3388ce6794e3, user_id=9d495a4106fa4a9eab8a2a8198ef9849, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:42.425462 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:42.429229 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:42.475211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1394/2758] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:42 2022] PUT /identity/v3/projects/695b71b9060343dab99d3388ce6794e3/users/9d495a4106fa4a9eab8a2a8198ef9849/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:42.486614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe566-0a25-4d6d-b5a4-ce378cd50fd8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.486941 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe566-0a25-4d6d-b5a4-ce378cd50fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.487484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe566-0a25-4d6d-b5a4-ce378cd50fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.531888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2cdfe566-0a25-4d6d-b5a4-ce378cd50fd8 None None] MFA Rules not processed for user `9d495a4106fa4a9eab8a2a8198ef9849`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:42.533091 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2cdfe566-0a25-4d6d-b5a4-ce378cd50fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:42.541831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:42.631964 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1365/2759] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:42.671762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:42.673728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:42.746253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f9a28-532e-40e1-8e96-246e49d07e2b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:42.750132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcb408-203a-4ee6-baff-4e517d7415f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:42.750842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcb408-203a-4ee6-baff-4e517d7415f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.751119 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcb408-203a-4ee6-baff-4e517d7415f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.751388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcb408-203a-4ee6-baff-4e517d7415f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.761450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcb408-203a-4ee6-baff-4e517d7415f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:42.763062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcb408-203a-4ee6-baff-4e517d7415f7 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:42.773978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1395/2760] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:42:42.789404 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfe566-0a25-4d6d-b5a4-ce378cd50fd8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:42.887939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfe566-0a25-4d6d-b5a4-ce378cd50fd8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:42.892579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:42.893204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.893477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.893744 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] PATH_INFO: `/v3/users/9d495a4106fa4a9eab8a2a8198ef9849` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:42.894894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d495a4106fa4a9eab8a2a8198ef9849)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:42.896779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:42.948649 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d495a4106fa4a9eab8a2a8198ef9849'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:42.956942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d495a4106fa4a9eab8a2a8198ef9849'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:42.957724 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d495a4106fa4a9eab8a2a8198ef9849'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:42.969602 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:42.970876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d495a4106fa4a9eab8a2a8198ef9849'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:42.971549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d495a4106fa4a9eab8a2a8198ef9849'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:42.986477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1366/2761] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:42 2022] DELETE /identity/v3/users/9d495a4106fa4a9eab8a2a8198ef9849 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:42.990947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b95d-3923-446a-a226-d46f723af9c1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:42.991232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b95d-3923-446a-a226-d46f723af9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:42.991501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b95d-3923-446a-a226-d46f723af9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:43.021551 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ffc3b95d-3923-446a-a226-d46f723af9c1 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:43.022791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ffc3b95d-3923-446a-a226-d46f723af9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:43.086672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1396/2762] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:43.096866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:43.098282 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:43.167383 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a0510-ee71-468a-8ca6-12a4ebbecd36 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:43.171222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57593258-4ac1-4309-beb0-f0db6d894b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:43.171814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57593258-4ac1-4309-beb0-f0db6d894b71 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:43.172087 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57593258-4ac1-4309-beb0-f0db6d894b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:43.172351 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57593258-4ac1-4309-beb0-f0db6d894b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:43.181213 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57593258-4ac1-4309-beb0-f0db6d894b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:43.182634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57593258-4ac1-4309-beb0-f0db6d894b71 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:43.193349 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1367/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:42:43.486906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b95d-3923-446a-a226-d46f723af9c1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:43.488806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ffc3b95d-3923-446a-a226-d46f723af9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:43.508486 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b95d-3923-446a-a226-d46f723af9c1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:43.515440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:43.516110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:43.516394 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:43.516669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] PATH_INFO: `/v3/projects/695b71b9060343dab99d3388ce6794e3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:43.517903 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=695b71b9060343dab99d3388ce6794e3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:43.519882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:43.575503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695b71b9060343dab99d3388ce6794e3', 'request_id': 'req-584a823b-57d4-419e-a655-5658e93b8177'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:43.585322 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695b71b9060343dab99d3388ce6794e3', 'request_id': 'req-584a823b-57d4-419e-a655-5658e93b8177'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:43.592639 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695b71b9060343dab99d3388ce6794e3', 'request_id': 'req-584a823b-57d4-419e-a655-5658e93b8177'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:43.595507 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:43.608611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] The token cache is being invalidate because project 695b71b9060343dab99d3388ce6794e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:43.609066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:43.610464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-584a823b-57d4-419e-a655-5658e93b8177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:43.612949 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1397/2764] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:43 2022] DELETE /identity/v3/projects/695b71b9060343dab99d3388ce6794e3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:43.620041 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687117-5161-45d4-a84c-1465413fadf4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:43.620405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687117-5161-45d4-a84c-1465413fadf4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:43.620780 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687117-5161-45d4-a84c-1465413fadf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:43.688772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ba687117-5161-45d4-a84c-1465413fadf4 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:43.690001 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ba687117-5161-45d4-a84c-1465413fadf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:43.771665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1368/2765] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:43.780736 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f813-f608-4302-b9ae-1fd7e3f49690 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:43.781110 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f813-f608-4302-b9ae-1fd7e3f49690 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:43.781863 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f813-f608-4302-b9ae-1fd7e3f49690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:43.807587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1735f813-f608-4302-b9ae-1fd7e3f49690 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:43.808585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1735f813-f608-4302-b9ae-1fd7e3f49690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:43.817188 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1398/2766] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:42:43.821563 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba687117-5161-45d4-a84c-1465413fadf4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:43.835314 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba687117-5161-45d4-a84c-1465413fadf4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:43.840741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:43.841359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:43.841631 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:43.841928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:43.843042 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:42:43.843193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:43.846200 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:43.867263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1369/2767] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:42:43.872685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735f813-f608-4302-b9ae-1fd7e3f49690 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:43.886416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735f813-f608-4302-b9ae-1fd7e3f49690 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:43.892028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff0921-d95c-4e50-9c12-38b7f9557cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:43.892628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0921-d95c-4e50-9c12-38b7f9557cda admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:43.892912 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0921-d95c-4e50-9c12-38b7f9557cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:43.893203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0921-d95c-4e50-9c12-38b7f9557cda admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:43.893847 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff0921-d95c-4e50-9c12-38b7f9557cda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:43.895827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff0921-d95c-4e50-9c12-38b7f9557cda admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:43.938117 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1399/2768] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:42:43 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:43.943758 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:43.957560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e2d83-3732-4e6d-803c-283daecd036c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:44.015334 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:44.034186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c0005-4238-4406-b444-207e99fe906d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:44.034821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c0005-4238-4406-b444-207e99fe906d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:44.043720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c0005-4238-4406-b444-207e99fe906d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:44.043720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c0005-4238-4406-b444-207e99fe906d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:44.043720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c0005-4238-4406-b444-207e99fe906d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:44.043720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c0005-4238-4406-b444-207e99fe906d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:44.074322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1370/2769] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:42:43 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:42:44.077802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff0921-d95c-4e50-9c12-38b7f9557cda admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:44.090418 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff0921-d95c-4e50-9c12-38b7f9557cda admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:44.096965 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56de8-ee04-4f8b-8b07-818239704d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:44.098514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56de8-ee04-4f8b-8b07-818239704d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:44.098514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56de8-ee04-4f8b-8b07-818239704d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:44.098738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56de8-ee04-4f8b-8b07-818239704d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:44.099426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc56de8-ee04-4f8b-8b07-818239704d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:44.102156 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc56de8-ee04-4f8b-8b07-818239704d0b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:44.121926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1400/2770] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:42:44.126401 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c0005-4238-4406-b444-207e99fe906d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:44.142471 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c0005-4238-4406-b444-207e99fe906d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:44.148190 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856708f-8d70-4130-bb29-3f4b9e7e346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:44.148808 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856708f-8d70-4130-bb29-3f4b9e7e346c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:44.149086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856708f-8d70-4130-bb29-3f4b9e7e346c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:44.149361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856708f-8d70-4130-bb29-3f4b9e7e346c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:44.150101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856708f-8d70-4130-bb29-3f4b9e7e346c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:44.151720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856708f-8d70-4130-bb29-3f4b9e7e346c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:44.165558 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1371/2771] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:42:44.167287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56de8-ee04-4f8b-8b07-818239704d0b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:44.178171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56de8-ee04-4f8b-8b07-818239704d0b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:44.182954 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:44.183630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:44.183908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:44.184178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] PATH_INFO: `/v3/projects/4cd84e814af142d2bbe4d98eb2f6b4f4/users/8ea10b8ce1da41108b2f922f60ca6a19/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:44.206056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd84e814af142d2bbe4d98eb2f6b4f4, user_id=8ea10b8ce1da41108b2f922f60ca6a19, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:44.211369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:44.234125 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1401/2772] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:42:44 2022] PUT /identity/v3/projects/4cd84e814af142d2bbe4d98eb2f6b4f4/users/8ea10b8ce1da41108b2f922f60ca6a19/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:44.246976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe424a4b-09e9-441d-ae1a-94976228f232 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:44.247283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe424a4b-09e9-441d-ae1a-94976228f232 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:44.247806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe424a4b-09e9-441d-ae1a-94976228f232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:44.280697 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fe424a4b-09e9-441d-ae1a-94976228f232 None None] MFA Rules not processed for user `8ea10b8ce1da41108b2f922f60ca6a19`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:44.282622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe424a4b-09e9-441d-ae1a-94976228f232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:44.376946 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1372/2773] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:42:44.384530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:44.386500 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:44.459674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d5470-fc1d-449c-bea6-9a0d96dd9b01 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:44.463634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b0dbe-22ba-4128-a722-ccdd5eca8575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:44.464251 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b0dbe-22ba-4128-a722-ccdd5eca8575 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:44.464523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b0dbe-22ba-4128-a722-ccdd5eca8575 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:44.464793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b0dbe-22ba-4128-a722-ccdd5eca8575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:44.474282 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b0dbe-22ba-4128-a722-ccdd5eca8575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:44.475764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b0dbe-22ba-4128-a722-ccdd5eca8575 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:44.489774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1402/2774] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:47.416536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe424a4b-09e9-441d-ae1a-94976228f232 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:47.494888 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe424a4b-09e9-441d-ae1a-94976228f232 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:47.500216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:47.500841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:47.501142 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:47.501409 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] PATH_INFO: `/v3/users/8ea10b8ce1da41108b2f922f60ca6a19` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:47.502710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea10b8ce1da41108b2f922f60ca6a19)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:47.504895 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:47.562196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea10b8ce1da41108b2f922f60ca6a19'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:47.573552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea10b8ce1da41108b2f922f60ca6a19'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:47.574204 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea10b8ce1da41108b2f922f60ca6a19'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:47.577255 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:47.592816 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea10b8ce1da41108b2f922f60ca6a19'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:47.593588 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea10b8ce1da41108b2f922f60ca6a19'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:47.608090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1373/2775] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:42:47 2022] DELETE /identity/v3/users/8ea10b8ce1da41108b2f922f60ca6a19 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:47.613092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20ad42-6120-4894-8de6-4b36c1a28646 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:47.613982 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20ad42-6120-4894-8de6-4b36c1a28646 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:47.614534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20ad42-6120-4894-8de6-4b36c1a28646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:47.634587 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bf20ad42-6120-4894-8de6-4b36c1a28646 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:47.635829 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bf20ad42-6120-4894-8de6-4b36c1a28646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:47.703013 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1403/2776] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:42:47.713242 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:47.714856 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:47.773906 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc289-444b-4e1a-a4f8-82a3c8122e70 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:47.777821 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:47.778435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:47.778739 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:47.779021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:47.789053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:47.790491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:47.801056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1374/2777] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:48.083440 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20ad42-6120-4894-8de6-4b36c1a28646 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:48.085104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bf20ad42-6120-4894-8de6-4b36c1a28646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:48.098193 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20ad42-6120-4894-8de6-4b36c1a28646 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:48.103604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:48.104441 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:48.104729 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:48.104978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] PATH_INFO: `/v3/projects/4cd84e814af142d2bbe4d98eb2f6b4f4` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:48.106286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd84e814af142d2bbe4d98eb2f6b4f4)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:48.108355 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:48.150437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:48.173270 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd84e814af142d2bbe4d98eb2f6b4f4', 'request_id': 'req-31e43708-6601-4793-8ff1-31c666b04a46'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:48.184074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd84e814af142d2bbe4d98eb2f6b4f4', 'request_id': 'req-31e43708-6601-4793-8ff1-31c666b04a46'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:48.191370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd84e814af142d2bbe4d98eb2f6b4f4', 'request_id': 'req-31e43708-6601-4793-8ff1-31c666b04a46'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:48.204974 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] The token cache is being invalidate because project 4cd84e814af142d2bbe4d98eb2f6b4f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:42:48.205263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:48.206562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-31e43708-6601-4793-8ff1-31c666b04a46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:42:48.209205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1404/2778] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:42:48 2022] DELETE /identity/v3/projects/4cd84e814af142d2bbe4d98eb2f6b4f4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:42:53.755518 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:53.757050 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:53.863100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886713a-5322-4d07-bbe0-cddc858033de service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:53.922813 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:42:53.932033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:53.932652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:53.932929 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:53.933198 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:53.935247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:53.947133 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:53.948726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:53.959706 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1375/2779] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:42:56.874670 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f5339-9666-4dd1-9e72-51e8c9974ef0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:56.875474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f5339-9666-4dd1-9e72-51e8c9974ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:56.875474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f5339-9666-4dd1-9e72-51e8c9974ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:56.896316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9d4f5339-9666-4dd1-9e72-51e8c9974ef0 None None] MFA Rules not processed for user `9f4dcda6ad1d401a9aed637906563d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:42:56.897777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9d4f5339-9666-4dd1-9e72-51e8c9974ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:56.953636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1405/2780] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:42:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:42:59.173410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:42:59.176235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:59.240281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee58a16-2d8a-4ceb-9778-9d34c28c541e service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:42:59.244594 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110a83b-fe51-4cad-bd43-9088c0333963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:42:59.245207 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a83b-fe51-4cad-bd43-9088c0333963 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:42:59.245477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a83b-fe51-4cad-bd43-9088c0333963 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:42:59.245740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a83b-fe51-4cad-bd43-9088c0333963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:42:59.247927 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d110a83b-fe51-4cad-bd43-9088c0333963 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:42:59.312268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110a83b-fe51-4cad-bd43-9088c0333963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:42:59.314804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110a83b-fe51-4cad-bd43-9088c0333963 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:42:59.330384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1376/2781] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:42:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:43:06.947212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=6,keystone:SELECT=39,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:43:09.323227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=55 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:43:59.886372 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daee88-b038-4e67-966f-47b8be2c156e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:43:59.886952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daee88-b038-4e67-966f-47b8be2c156e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:43:59.887890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daee88-b038-4e67-966f-47b8be2c156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:43:59.909573 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f6daee88-b038-4e67-966f-47b8be2c156e None None] MFA Rules not processed for user `c11d44667e8e4db5bccccaee16d89a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:43:59.910839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f6daee88-b038-4e67-966f-47b8be2c156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:43:59.968517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1406/2782] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:43:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:44:03.506972 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b07f1-5818-49c6-86c3-1a5031445ba7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:44:03.507500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b07f1-5818-49c6-86c3-1a5031445ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:44:03.507603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b07f1-5818-49c6-86c3-1a5031445ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:44:03.527632 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-157b07f1-5818-49c6-86c3-1a5031445ba7 None None] MFA Rules not processed for user `9f4dcda6ad1d401a9aed637906563d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:44:03.528822 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-157b07f1-5818-49c6-86c3-1a5031445ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:44:03.535913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1377/2783] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:44:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:44:09.961748 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=25 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:44:13.521942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:44:56.985806 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6daee88-b038-4e67-966f-47b8be2c156e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:44:56.987343 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f6daee88-b038-4e67-966f-47b8be2c156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:44:57.075993 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6daee88-b038-4e67-966f-47b8be2c156e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:44:57.080076 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:44:57.080686 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:44:57.080962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:44:57.081236 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:44:57.083050 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:44:57.178874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:44:57.180191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:44:57.203427 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1407/2784] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:44:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:44:59.768855 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b07f1-5818-49c6-86c3-1a5031445ba7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:44:59.789402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b07f1-5818-49c6-86c3-1a5031445ba7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:44:59.793582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:44:59.794205 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:44:59.794478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:44:59.794743 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:44:59.796433 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:44:59.810124 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:44:59.811564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:44:59.823264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1378/2785] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:44:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:45:07.200101 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=75 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:08.930664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:08.932246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:08.957092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a403b-3e83-4d94-a9e2-dbf1a0ae6790 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:08.960846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:08.961445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:08.961725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:08.961995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:08.963868 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:08.974252 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:08.975725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:08.984064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1408/2786] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:09.817715 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:12.222690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:12.224428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:12.319573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510f622-3eff-407f-9f92-aa86718fb82c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:12.325269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:12.325985 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:12.326268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:12.326664 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] PATH_INFO: `/v3/users/9f4dcda6ad1d401a9aed637906563d72` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:12.328016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f4dcda6ad1d401a9aed637906563d72)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:12.330250 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:12.392844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f4dcda6ad1d401a9aed637906563d72'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.402825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f4dcda6ad1d401a9aed637906563d72'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.403496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f4dcda6ad1d401a9aed637906563d72'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.426639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f4dcda6ad1d401a9aed637906563d72'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.427405 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f4dcda6ad1d401a9aed637906563d72'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.441065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1379/2787] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:12 2022] DELETE /identity/v3/users/9f4dcda6ad1d401a9aed637906563d72 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:12.497197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:12.498539 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:12.513997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b0338-1cd1-4b06-aa3c-2ac3ad17cf6d service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:12.574290 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec506484-c3bf-4454-b33a-e716113a3a85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:12.574908 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec506484-c3bf-4454-b33a-e716113a3a85 service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:12.575184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec506484-c3bf-4454-b33a-e716113a3a85 service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:12.575449 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec506484-c3bf-4454-b33a-e716113a3a85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:12.577149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ec506484-c3bf-4454-b33a-e716113a3a85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:12.588561 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec506484-c3bf-4454-b33a-e716113a3a85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:12.590363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec506484-c3bf-4454-b33a-e716113a3a85 service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:12.600732 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1409/2788] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:12.650359 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:12.651944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:12.721951 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af98ec0-a0ce-4d45-8717-d7ef0ee3d73d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:12.727082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:12.727710 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:12.728075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:12.729049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] PATH_INFO: `/v3/projects/ae904e4b1df846e497cdf0edc46edaf1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:12.730134 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae904e4b1df846e497cdf0edc46edaf1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:12.732279 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:12.756568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:12.782866 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae904e4b1df846e497cdf0edc46edaf1', 'request_id': 'req-b3bcac33-0261-4d91-b3f3-6bb73f358145'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.789146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae904e4b1df846e497cdf0edc46edaf1', 'request_id': 'req-b3bcac33-0261-4d91-b3f3-6bb73f358145'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.797539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae904e4b1df846e497cdf0edc46edaf1', 'request_id': 'req-b3bcac33-0261-4d91-b3f3-6bb73f358145'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.809247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] The token cache is being invalidate because project ae904e4b1df846e497cdf0edc46edaf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:12.809639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.810288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3bcac33-0261-4d91-b3f3-6bb73f358145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:12.815180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1380/2789] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:12 2022] DELETE /identity/v3/projects/ae904e4b1df846e497cdf0edc46edaf1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:45:12.817336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296e7ab-4e70-443c-b68a-dd3ed6f20e80 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:12.817627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296e7ab-4e70-443c-b68a-dd3ed6f20e80 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:12.818065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296e7ab-4e70-443c-b68a-dd3ed6f20e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:12.838595 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b296e7ab-4e70-443c-b68a-dd3ed6f20e80 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:12.839836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b296e7ab-4e70-443c-b68a-dd3ed6f20e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:12.903887 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1410/2790] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:12.911656 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750b70-9730-41bb-bf05-9ceaa6c0e136 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:12.911937 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750b70-9730-41bb-bf05-9ceaa6c0e136 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:12.912208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750b70-9730-41bb-bf05-9ceaa6c0e136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:12.933418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4750b70-9730-41bb-bf05-9ceaa6c0e136 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:12.934625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4750b70-9730-41bb-bf05-9ceaa6c0e136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:12.944415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1381/2791] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:12.949875 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296e7ab-4e70-443c-b68a-dd3ed6f20e80 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:12.960390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296e7ab-4e70-443c-b68a-dd3ed6f20e80 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:12.965284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:12.965978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:12.966253 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:12.966545 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:12.967718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:45:12.967718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:12.969167 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:12.982742 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1411/2792] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:45:12.986030 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4750b70-9730-41bb-bf05-9ceaa6c0e136 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:12.995159 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4750b70-9730-41bb-bf05-9ceaa6c0e136 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:12.999880 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216fcc1-b193-4aaa-8b36-7681ffef7712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:13.000476 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216fcc1-b193-4aaa-8b36-7681ffef7712 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:13.000877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216fcc1-b193-4aaa-8b36-7681ffef7712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:13.001005 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216fcc1-b193-4aaa-8b36-7681ffef7712 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:13.001631 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216fcc1-b193-4aaa-8b36-7681ffef7712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:13.003547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216fcc1-b193-4aaa-8b36-7681ffef7712 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:13.041716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1382/2793] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:12 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:13.045992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:13.057245 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b1530-dba2-4c8e-a99b-d004163e880e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:13.112920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:13.123074 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaba5f5-fcb6-4017-8183-b7b3cbf555f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:13.123692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaba5f5-fcb6-4017-8183-b7b3cbf555f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:13.123969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaba5f5-fcb6-4017-8183-b7b3cbf555f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:13.124240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaba5f5-fcb6-4017-8183-b7b3cbf555f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:13.124884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaba5f5-fcb6-4017-8183-b7b3cbf555f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:13.126861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaba5f5-fcb6-4017-8183-b7b3cbf555f3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:13.149952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1412/2794] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:13 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:13.154322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216fcc1-b193-4aaa-8b36-7681ffef7712 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:13.164802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216fcc1-b193-4aaa-8b36-7681ffef7712 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:13.169550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4d6ee-e58b-4ee2-b510-0d2a599521b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:13.170143 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4d6ee-e58b-4ee2-b510-0d2a599521b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:13.170417 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4d6ee-e58b-4ee2-b510-0d2a599521b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:13.170682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4d6ee-e58b-4ee2-b510-0d2a599521b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:13.171364 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4d6ee-e58b-4ee2-b510-0d2a599521b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:13.172956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4d6ee-e58b-4ee2-b510-0d2a599521b5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:13.183500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1383/2795] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:13.187566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaba5f5-fcb6-4017-8183-b7b3cbf555f3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:13.198677 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaba5f5-fcb6-4017-8183-b7b3cbf555f3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:13.203976 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8db4d-1ed6-45a2-868d-0ac4d20f677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:13.204642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8db4d-1ed6-45a2-868d-0ac4d20f677c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:13.204924 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8db4d-1ed6-45a2-868d-0ac4d20f677c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:13.205199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8db4d-1ed6-45a2-868d-0ac4d20f677c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:13.205838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8db4d-1ed6-45a2-868d-0ac4d20f677c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:13.207485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8db4d-1ed6-45a2-868d-0ac4d20f677c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:13.219320 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1413/2796] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:13.224090 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4d6ee-e58b-4ee2-b510-0d2a599521b5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:13.236603 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4d6ee-e58b-4ee2-b510-0d2a599521b5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:13.242328 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:13.243008 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:13.243277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:13.243545 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] PATH_INFO: `/v3/projects/09961741edfa42288307bfe2f406578d/users/201721f1cf9041619f04df9304ceb736/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:13.260182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09961741edfa42288307bfe2f406578d, user_id=201721f1cf9041619f04df9304ceb736, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:13.262854 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:13.278273 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1384/2797] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:13 2022] PUT /identity/v3/projects/09961741edfa42288307bfe2f406578d/users/201721f1cf9041619f04df9304ceb736/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:13.289609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da1df-dfee-4d6f-8790-15c024776d87 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:13.289900 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da1df-dfee-4d6f-8790-15c024776d87 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:13.290174 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da1df-dfee-4d6f-8790-15c024776d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:13.320799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-015da1df-dfee-4d6f-8790-15c024776d87 None None] MFA Rules not processed for user `201721f1cf9041619f04df9304ceb736`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:13.322028 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-015da1df-dfee-4d6f-8790-15c024776d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:13.387494 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1414/2798] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:13.398642 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:13.400457 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:13.478262 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0aabb-a81c-49b0-96cc-9fa92e2a09f5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:13.484362 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c137d2-be06-4672-866e-d2a95426f86d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:13.484967 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c137d2-be06-4672-866e-d2a95426f86d service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:13.485239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c137d2-be06-4672-866e-d2a95426f86d service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:13.485506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c137d2-be06-4672-866e-d2a95426f86d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:13.493450 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c137d2-be06-4672-866e-d2a95426f86d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:13.495195 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c137d2-be06-4672-866e-d2a95426f86d service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:13.504427 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1385/2799] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:20.895481 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015da1df-dfee-4d6f-8790-15c024776d87 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:20.965036 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015da1df-dfee-4d6f-8790-15c024776d87 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:20.969738 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:20.970376 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:20.970652 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:20.970951 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] PATH_INFO: `/v3/users/201721f1cf9041619f04df9304ceb736` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:20.972064 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201721f1cf9041619f04df9304ceb736)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:20.973998 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:21.155845 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201721f1cf9041619f04df9304ceb736'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.164226 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201721f1cf9041619f04df9304ceb736'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.164883 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201721f1cf9041619f04df9304ceb736'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.172445 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:21.178408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201721f1cf9041619f04df9304ceb736'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.179470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201721f1cf9041619f04df9304ceb736'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.192855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1415/2800] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:20 2022] DELETE /identity/v3/users/201721f1cf9041619f04df9304ceb736 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:21.197506 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86e235-27af-4a93-9abd-e90908467df8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:21.198000 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86e235-27af-4a93-9abd-e90908467df8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:21.198173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86e235-27af-4a93-9abd-e90908467df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:21.220322 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ac86e235-27af-4a93-9abd-e90908467df8 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:21.221538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ac86e235-27af-4a93-9abd-e90908467df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:21.246274 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:21.290772 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1386/2801] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:45:21.300852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:21.302498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:21.366431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967f51b-aef3-4c30-a599-fe464854e5e2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:21.370368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b519c5-f8bb-4a87-adfa-2d9c4a1fa718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:21.371018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b519c5-f8bb-4a87-adfa-2d9c4a1fa718 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:21.371295 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b519c5-f8bb-4a87-adfa-2d9c4a1fa718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:21.371564 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b519c5-f8bb-4a87-adfa-2d9c4a1fa718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:21.381447 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b519c5-f8bb-4a87-adfa-2d9c4a1fa718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:21.383004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b519c5-f8bb-4a87-adfa-2d9c4a1fa718 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:21.393579 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1416/2802] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:21.669779 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86e235-27af-4a93-9abd-e90908467df8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:21.671240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ac86e235-27af-4a93-9abd-e90908467df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:21.683498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86e235-27af-4a93-9abd-e90908467df8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:21.689456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:21.690176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:21.690515 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:21.690877 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] PATH_INFO: `/v3/projects/09961741edfa42288307bfe2f406578d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:21.692082 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09961741edfa42288307bfe2f406578d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:21.694410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:21.742831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09961741edfa42288307bfe2f406578d', 'request_id': 'req-06347522-b653-4b5c-a0cc-561835b49d56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.749621 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09961741edfa42288307bfe2f406578d', 'request_id': 'req-06347522-b653-4b5c-a0cc-561835b49d56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.758823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09961741edfa42288307bfe2f406578d', 'request_id': 'req-06347522-b653-4b5c-a0cc-561835b49d56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.769434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] The token cache is being invalidate because project 09961741edfa42288307bfe2f406578d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:21.769712 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.770344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06347522-b653-4b5c-a0cc-561835b49d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:21.771908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1387/2803] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:21 2022] DELETE /identity/v3/projects/09961741edfa42288307bfe2f406578d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:21.777002 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefac40-b718-46b8-a256-313646ab1219 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:21.777284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefac40-b718-46b8-a256-313646ab1219 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:21.777565 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefac40-b718-46b8-a256-313646ab1219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:21.797058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1eefac40-b718-46b8-a256-313646ab1219 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:21.798304 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1eefac40-b718-46b8-a256-313646ab1219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:21.858472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1417/2804] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:21.866353 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41e9ce-bf94-4e3b-a751-a40f81aeffa9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:21.866634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41e9ce-bf94-4e3b-a751-a40f81aeffa9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:21.866930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41e9ce-bf94-4e3b-a751-a40f81aeffa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:21.887065 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3a41e9ce-bf94-4e3b-a751-a40f81aeffa9 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:21.888302 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a41e9ce-bf94-4e3b-a751-a40f81aeffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:21.896551 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1388/2805] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:21.901359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefac40-b718-46b8-a256-313646ab1219 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:21.911155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefac40-b718-46b8-a256-313646ab1219 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:21.915841 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:21.916521 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:21.916958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:21.917300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:21.918022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:45:21.918323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:21.919955 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:21.932085 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1418/2806] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:45:21.936752 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41e9ce-bf94-4e3b-a751-a40f81aeffa9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:21.946606 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41e9ce-bf94-4e3b-a751-a40f81aeffa9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:21.952473 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81123d86-1887-4284-9cb6-d8fcca176717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:21.953081 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81123d86-1887-4284-9cb6-d8fcca176717 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:21.953357 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81123d86-1887-4284-9cb6-d8fcca176717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:21.953623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81123d86-1887-4284-9cb6-d8fcca176717 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:21.954277 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81123d86-1887-4284-9cb6-d8fcca176717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:21.956708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81123d86-1887-4284-9cb6-d8fcca176717 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:21.985407 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1389/2807] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:21 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:21.989705 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:22.000318 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06341-e274-4c2d-97fd-46e4f15bbc26 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:22.056657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faaba4-2671-467b-8f03-071cd4ebf6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:22.057293 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faaba4-2671-467b-8f03-071cd4ebf6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:22.057562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faaba4-2671-467b-8f03-071cd4ebf6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:22.057827 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faaba4-2671-467b-8f03-071cd4ebf6e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:22.058537 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faaba4-2671-467b-8f03-071cd4ebf6e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:22.060513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faaba4-2671-467b-8f03-071cd4ebf6e0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:22.078405 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:22.084288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1419/2808] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:21 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:45:22.088771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81123d86-1887-4284-9cb6-d8fcca176717 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:22.099829 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81123d86-1887-4284-9cb6-d8fcca176717 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:22.112281 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14b71a-39b5-463d-8e32-ea83bd07ebd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:22.112894 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14b71a-39b5-463d-8e32-ea83bd07ebd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:22.113165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14b71a-39b5-463d-8e32-ea83bd07ebd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:22.113428 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14b71a-39b5-463d-8e32-ea83bd07ebd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:22.114074 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14b71a-39b5-463d-8e32-ea83bd07ebd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:22.115719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14b71a-39b5-463d-8e32-ea83bd07ebd8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:22.126690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1390/2809] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:22.131123 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faaba4-2671-467b-8f03-071cd4ebf6e0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:22.141478 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faaba4-2671-467b-8f03-071cd4ebf6e0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:22.146073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475bbfa-60ac-4b0e-940c-f8d9a665533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:22.146676 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475bbfa-60ac-4b0e-940c-f8d9a665533a admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:22.146978 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475bbfa-60ac-4b0e-940c-f8d9a665533a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:22.147248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475bbfa-60ac-4b0e-940c-f8d9a665533a admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:22.147995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475bbfa-60ac-4b0e-940c-f8d9a665533a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:22.149653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475bbfa-60ac-4b0e-940c-f8d9a665533a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:22.160932 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1420/2810] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:22.165348 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14b71a-39b5-463d-8e32-ea83bd07ebd8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:22.175812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14b71a-39b5-463d-8e32-ea83bd07ebd8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:22.180508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:22.181157 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:22.181434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:22.181703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] PATH_INFO: `/v3/projects/4def0b55b59d482d9cc859d8de7bec71/users/afafffa4f17d4cf3946feba65fa9ce21/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:22.198410 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4def0b55b59d482d9cc859d8de7bec71, user_id=afafffa4f17d4cf3946feba65fa9ce21, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:22.201193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:22.214893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1391/2811] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:22 2022] PUT /identity/v3/projects/4def0b55b59d482d9cc859d8de7bec71/users/afafffa4f17d4cf3946feba65fa9ce21/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:45:22.227882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea08d-75a5-466f-922c-9fdb278ac0f3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:22.228411 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea08d-75a5-466f-922c-9fdb278ac0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:22.228795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea08d-75a5-466f-922c-9fdb278ac0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:22.263395 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-627ea08d-75a5-466f-922c-9fdb278ac0f3 None None] MFA Rules not processed for user `afafffa4f17d4cf3946feba65fa9ce21`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:22.264754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-627ea08d-75a5-466f-922c-9fdb278ac0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:22.343043 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1421/2812] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:45:22.354330 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:22.356481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:22.408239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:22.422570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d9d1b-cc10-467f-ba02-8172bf943cab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:22.426803 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826eb6ea-755d-49b0-a540-85763ff1f717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:22.427415 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826eb6ea-755d-49b0-a540-85763ff1f717 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:22.427689 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826eb6ea-755d-49b0-a540-85763ff1f717 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:22.427957 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826eb6ea-755d-49b0-a540-85763ff1f717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:22.436893 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826eb6ea-755d-49b0-a540-85763ff1f717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:22.438331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826eb6ea-755d-49b0-a540-85763ff1f717 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:22.447379 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1392/2813] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:30.974313 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea08d-75a5-466f-922c-9fdb278ac0f3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:31.054370 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea08d-75a5-466f-922c-9fdb278ac0f3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:31.061291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:31.062634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:31.063145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:31.063620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] PATH_INFO: `/v3/users/afafffa4f17d4cf3946feba65fa9ce21` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:31.065105 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] RBAC: Authorizing `identity:delete_user(user_id=afafffa4f17d4cf3946feba65fa9ce21)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:31.067149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:31.118655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afafffa4f17d4cf3946feba65fa9ce21'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.128383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afafffa4f17d4cf3946feba65fa9ce21'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.129018 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afafffa4f17d4cf3946feba65fa9ce21'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.141995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afafffa4f17d4cf3946feba65fa9ce21'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.142685 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afafffa4f17d4cf3946feba65fa9ce21'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.151629 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:31.157177 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1422/2814] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:30 2022] DELETE /identity/v3/users/afafffa4f17d4cf3946feba65fa9ce21 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:31.162060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ccc4-95e2-4a51-aec6-4a7a7834d66c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:31.162540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ccc4-95e2-4a51-aec6-4a7a7834d66c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:31.162658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ccc4-95e2-4a51-aec6-4a7a7834d66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:31.181947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-33a5ccc4-95e2-4a51-aec6-4a7a7834d66c None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:31.183209 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-33a5ccc4-95e2-4a51-aec6-4a7a7834d66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:31.245456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1393/2815] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:31.255616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:31.257160 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:31.324950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722d48e-a36d-4e6a-8459-22a6a736863d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:31.329664 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc8c90-0c07-4250-9443-d406fb0b5d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:31.330284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8c90-0c07-4250-9443-d406fb0b5d66 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:31.330562 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8c90-0c07-4250-9443-d406fb0b5d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:31.330865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8c90-0c07-4250-9443-d406fb0b5d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:31.341866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc8c90-0c07-4250-9443-d406fb0b5d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:31.343368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc8c90-0c07-4250-9443-d406fb0b5d66 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:31.353988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1423/2816] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:31.648961 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5ccc4-95e2-4a51-aec6-4a7a7834d66c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:31.650478 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-33a5ccc4-95e2-4a51-aec6-4a7a7834d66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:31.663101 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5ccc4-95e2-4a51-aec6-4a7a7834d66c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:31.669086 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:31.669738 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:31.670007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:31.670268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] PATH_INFO: `/v3/projects/4def0b55b59d482d9cc859d8de7bec71` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:31.671558 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4def0b55b59d482d9cc859d8de7bec71)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:31.673507 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:31.724344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4def0b55b59d482d9cc859d8de7bec71', 'request_id': 'req-e2d09317-5f0a-4663-a010-19b21443eb67'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.732099 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4def0b55b59d482d9cc859d8de7bec71', 'request_id': 'req-e2d09317-5f0a-4663-a010-19b21443eb67'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.741601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4def0b55b59d482d9cc859d8de7bec71', 'request_id': 'req-e2d09317-5f0a-4663-a010-19b21443eb67'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.753006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] The token cache is being invalidate because project 4def0b55b59d482d9cc859d8de7bec71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:31.753320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.754015 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e2d09317-5f0a-4663-a010-19b21443eb67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:31.755659 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1394/2817] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:31 2022] DELETE /identity/v3/projects/4def0b55b59d482d9cc859d8de7bec71 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:31.760768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ebb3-0167-4a13-8914-da6457d2a2ec None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:31.761055 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ebb3-0167-4a13-8914-da6457d2a2ec None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:31.761329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ebb3-0167-4a13-8914-da6457d2a2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:31.781621 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4551ebb3-0167-4a13-8914-da6457d2a2ec None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:31.782906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4551ebb3-0167-4a13-8914-da6457d2a2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:31.844166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1424/2818] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:31.852609 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602db09-e8c9-478a-900f-9f3542c90ee6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:31.852891 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602db09-e8c9-478a-900f-9f3542c90ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:31.853170 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602db09-e8c9-478a-900f-9f3542c90ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:31.877087 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7602db09-e8c9-478a-900f-9f3542c90ee6 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:31.878283 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7602db09-e8c9-478a-900f-9f3542c90ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:31.887899 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1395/2819] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:31.892546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551ebb3-0167-4a13-8914-da6457d2a2ec None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:31.902751 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551ebb3-0167-4a13-8914-da6457d2a2ec None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:31.908559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:31.909154 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:31.909426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:31.909691 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:31.910330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:45:31.910623 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:31.912213 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:31.923793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1425/2820] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:45:31.928197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602db09-e8c9-478a-900f-9f3542c90ee6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:31.938040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602db09-e8c9-478a-900f-9f3542c90ee6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:31.943973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9edfa-4ef6-4772-bbdf-d7b0ab7c500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:31.944562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9edfa-4ef6-4772-bbdf-d7b0ab7c500a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:31.944833 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9edfa-4ef6-4772-bbdf-d7b0ab7c500a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:31.945100 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9edfa-4ef6-4772-bbdf-d7b0ab7c500a admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:31.945717 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9edfa-4ef6-4772-bbdf-d7b0ab7c500a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:31.947674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9edfa-4ef6-4772-bbdf-d7b0ab7c500a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:31.977690 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1396/2821] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:31 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:31.982382 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:31.997437 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d36767-7444-4a3a-9b92-0ee70079300b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.066517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512a50d-07f3-4562-bfca-5671f8dab537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.067653 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512a50d-07f3-4562-bfca-5671f8dab537 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.067928 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512a50d-07f3-4562-bfca-5671f8dab537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.068197 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512a50d-07f3-4562-bfca-5671f8dab537 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.068854 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512a50d-07f3-4562-bfca-5671f8dab537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.071334 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512a50d-07f3-4562-bfca-5671f8dab537 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.096443 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1426/2822] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:31 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:32.100846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9edfa-4ef6-4772-bbdf-d7b0ab7c500a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.111533 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9edfa-4ef6-4772-bbdf-d7b0ab7c500a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.116526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05b716-db29-4223-9750-92e6d30d2949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.117161 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b716-db29-4223-9750-92e6d30d2949 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.117434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b716-db29-4223-9750-92e6d30d2949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.117702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b716-db29-4223-9750-92e6d30d2949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.118346 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05b716-db29-4223-9750-92e6d30d2949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.120084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05b716-db29-4223-9750-92e6d30d2949 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.131180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1397/2823] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:32.135347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512a50d-07f3-4562-bfca-5671f8dab537 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.145795 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512a50d-07f3-4562-bfca-5671f8dab537 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.150870 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15c882-00b0-4591-89f5-7d46f47332f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.151470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15c882-00b0-4591-89f5-7d46f47332f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.151745 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15c882-00b0-4591-89f5-7d46f47332f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.152017 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15c882-00b0-4591-89f5-7d46f47332f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.152636 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15c882-00b0-4591-89f5-7d46f47332f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.154406 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15c882-00b0-4591-89f5-7d46f47332f4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.164950 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1427/2824] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:32.169441 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05b716-db29-4223-9750-92e6d30d2949 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.180145 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05b716-db29-4223-9750-92e6d30d2949 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.187721 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0eb7e7-1330-4df3-a3c5-9c92bd55ef29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.188371 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb7e7-1330-4df3-a3c5-9c92bd55ef29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.188645 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb7e7-1330-4df3-a3c5-9c92bd55ef29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.188905 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eb7e7-1330-4df3-a3c5-9c92bd55ef29 admin admin] PATH_INFO: `/v3/projects/bab0ec2982604661b0e4d33d583b2f4f/users/2daf736df1404c3189c8fca2ba0445e9/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.204475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0eb7e7-1330-4df3-a3c5-9c92bd55ef29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab0ec2982604661b0e4d33d583b2f4f, user_id=2daf736df1404c3189c8fca2ba0445e9, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.207237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0eb7e7-1330-4df3-a3c5-9c92bd55ef29 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.222226 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1398/2825] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:32 2022] PUT /identity/v3/projects/bab0ec2982604661b0e4d33d583b2f4f/users/2daf736df1404c3189c8fca2ba0445e9/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:32.232878 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1540-1437-41f0-a1a4-5cd8b75cd01c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.233187 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1540-1437-41f0-a1a4-5cd8b75cd01c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.233464 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1540-1437-41f0-a1a4-5cd8b75cd01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.247846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:32.264436 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9fbc1540-1437-41f0-a1a4-5cd8b75cd01c None None] MFA Rules not processed for user `2daf736df1404c3189c8fca2ba0445e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:32.265675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9fbc1540-1437-41f0-a1a4-5cd8b75cd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:32.330611 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1428/2826] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:32 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:32.335385 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0eb7e7-1330-4df3-a3c5-9c92bd55ef29 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.342299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:32.403035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0eb7e7-1330-4df3-a3c5-9c92bd55ef29 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.408761 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696873-ac09-4b0a-b653-15909b3bacc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.409362 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696873-ac09-4b0a-b653-15909b3bacc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.409639 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696873-ac09-4b0a-b653-15909b3bacc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.409910 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae696873-ac09-4b0a-b653-15909b3bacc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.410531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae696873-ac09-4b0a-b653-15909b3bacc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.412468 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae696873-ac09-4b0a-b653-15909b3bacc7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.441502 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1399/2827] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:32 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:32.447958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc1540-1437-41f0-a1a4-5cd8b75cd01c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.461673 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc1540-1437-41f0-a1a4-5cd8b75cd01c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.534378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472e07c-7f5d-4d53-b5d5-117c415a2dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.535023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472e07c-7f5d-4d53-b5d5-117c415a2dde admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.535302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472e07c-7f5d-4d53-b5d5-117c415a2dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.535581 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472e07c-7f5d-4d53-b5d5-117c415a2dde admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.536203 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472e07c-7f5d-4d53-b5d5-117c415a2dde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.538140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472e07c-7f5d-4d53-b5d5-117c415a2dde admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.561968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1429/2828] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:32 2022] POST /identity/v3/users => generated 308 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:32.566247 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696873-ac09-4b0a-b653-15909b3bacc7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.577118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae696873-ac09-4b0a-b653-15909b3bacc7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.582411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d42432-4b00-4067-a941-39a4a8adebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.582979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d42432-4b00-4067-a941-39a4a8adebe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.583257 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d42432-4b00-4067-a941-39a4a8adebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.583526 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d42432-4b00-4067-a941-39a4a8adebe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.584175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d42432-4b00-4067-a941-39a4a8adebe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.585778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d42432-4b00-4067-a941-39a4a8adebe3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.596973 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1400/2829] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:32.603259 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472e07c-7f5d-4d53-b5d5-117c415a2dde admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.617820 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472e07c-7f5d-4d53-b5d5-117c415a2dde admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.625891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc9ac1-68db-4167-a3eb-1d90cf01eeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.626547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc9ac1-68db-4167-a3eb-1d90cf01eeb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.626865 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc9ac1-68db-4167-a3eb-1d90cf01eeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.627136 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc9ac1-68db-4167-a3eb-1d90cf01eeb8 admin admin] PATH_INFO: `/v3/projects/d770b6c9957c4627b9314385cd31f474/users/f87a956205624450a75f304dae31a89d/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.642431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc9ac1-68db-4167-a3eb-1d90cf01eeb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d770b6c9957c4627b9314385cd31f474, user_id=f87a956205624450a75f304dae31a89d, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.645104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc9ac1-68db-4167-a3eb-1d90cf01eeb8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.660161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1430/2830] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:32 2022] PUT /identity/v3/projects/d770b6c9957c4627b9314385cd31f474/users/f87a956205624450a75f304dae31a89d/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:32.664227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d42432-4b00-4067-a941-39a4a8adebe3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.674942 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d42432-4b00-4067-a941-39a4a8adebe3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.730660 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41feed-b015-4638-a90a-a73790e47415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.731332 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41feed-b015-4638-a90a-a73790e47415 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.731619 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41feed-b015-4638-a90a-a73790e47415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.731886 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41feed-b015-4638-a90a-a73790e47415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.732531 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41feed-b015-4638-a90a-a73790e47415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.734132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41feed-b015-4638-a90a-a73790e47415 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.743869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1401/2831] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:32.748426 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc9ac1-68db-4167-a3eb-1d90cf01eeb8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.758991 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc9ac1-68db-4167-a3eb-1d90cf01eeb8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:32.764591 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:32.765238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.765516 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.765788 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] PATH_INFO: `/v3/projects/d770b6c9957c4627b9314385cd31f474/users/f87a956205624450a75f304dae31a89d/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.767890 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d770b6c9957c4627b9314385cd31f474, user_id=f87a956205624450a75f304dae31a89d, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:32.770633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:32.781233 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1431/2832] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:32 2022] PUT /identity/v3/projects/d770b6c9957c4627b9314385cd31f474/users/f87a956205624450a75f304dae31a89d/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:32.846953 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d803a-6e8c-4010-933c-151e637f846b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:32.847188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d803a-6e8c-4010-933c-151e637f846b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:32.847394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d803a-6e8c-4010-933c-151e637f846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:32.884007 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-045d803a-6e8c-4010-933c-151e637f846b None None] MFA Rules not processed for user `f87a956205624450a75f304dae31a89d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:32.885203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-045d803a-6e8c-4010-933c-151e637f846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:32.958227 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:32.963384 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1402/2833] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:32.974640 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:32.976765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:33.035089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:33.039559 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42cf8b-f36e-414d-bc28-e27b9ea458b5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:33.044078 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73324b-036f-425f-81b2-197305bd09a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:33.044704 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73324b-036f-425f-81b2-197305bd09a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:33.045173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73324b-036f-425f-81b2-197305bd09a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:33.045503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73324b-036f-425f-81b2-197305bd09a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:33.055397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa73324b-036f-425f-81b2-197305bd09a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:33.056874 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa73324b-036f-425f-81b2-197305bd09a2 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:33.068129 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1432/2834] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:32 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:33.225965 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d803a-6e8c-4010-933c-151e637f846b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:33.291495 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d803a-6e8c-4010-933c-151e637f846b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:33.296867 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:33.297517 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:33.297789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:33.298053 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] PATH_INFO: `/v3/users/2daf736df1404c3189c8fca2ba0445e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:33.299237 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2daf736df1404c3189c8fca2ba0445e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:33.301411 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:33.352104 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2daf736df1404c3189c8fca2ba0445e9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.361307 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2daf736df1404c3189c8fca2ba0445e9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.361916 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2daf736df1404c3189c8fca2ba0445e9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.374413 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2daf736df1404c3189c8fca2ba0445e9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.375181 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2daf736df1404c3189c8fca2ba0445e9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.388767 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1403/2835] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:33 2022] DELETE /identity/v3/users/2daf736df1404c3189c8fca2ba0445e9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:33.395930 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff95cb-8b2c-496b-a9ee-d013be5e1016 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:33.396315 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff95cb-8b2c-496b-a9ee-d013be5e1016 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:33.396674 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff95cb-8b2c-496b-a9ee-d013be5e1016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:33.423254 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-43ff95cb-8b2c-496b-a9ee-d013be5e1016 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:33.424485 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-43ff95cb-8b2c-496b-a9ee-d013be5e1016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:33.495247 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1433/2836] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:33.505270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:33.506834 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:33.570874 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0ea06-4476-42ba-b834-3015348ab8e3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:33.575377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e214f-cde8-40c9-8b5b-492d3d6caf66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:33.575993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e214f-cde8-40c9-8b5b-492d3d6caf66 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:33.576239 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e214f-cde8-40c9-8b5b-492d3d6caf66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:33.576536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e214f-cde8-40c9-8b5b-492d3d6caf66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:33.586139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e214f-cde8-40c9-8b5b-492d3d6caf66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:33.587593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e214f-cde8-40c9-8b5b-492d3d6caf66 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:33.597289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1404/2837] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:33.897661 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff95cb-8b2c-496b-a9ee-d013be5e1016 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:33.899284 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-43ff95cb-8b2c-496b-a9ee-d013be5e1016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:33.912392 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff95cb-8b2c-496b-a9ee-d013be5e1016 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:33.917774 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:33.918419 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:33.918759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:33.919032 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] PATH_INFO: `/v3/projects/bab0ec2982604661b0e4d33d583b2f4f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:33.920356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab0ec2982604661b0e4d33d583b2f4f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:33.922291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:33.971027 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab0ec2982604661b0e4d33d583b2f4f', 'request_id': 'req-5997db89-bfb3-433a-9e53-69572154df80'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.979319 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab0ec2982604661b0e4d33d583b2f4f', 'request_id': 'req-5997db89-bfb3-433a-9e53-69572154df80'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.985513 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab0ec2982604661b0e4d33d583b2f4f', 'request_id': 'req-5997db89-bfb3-433a-9e53-69572154df80'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.996015 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] The token cache is being invalidate because project bab0ec2982604661b0e4d33d583b2f4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:33.996296 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.997127 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:33.998594 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1434/2838] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:33 2022] DELETE /identity/v3/projects/bab0ec2982604661b0e4d33d583b2f4f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:34.002954 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e214f-cde8-40c9-8b5b-492d3d6caf66 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:34.014132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e214f-cde8-40c9-8b5b-492d3d6caf66 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:34.034768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:34.079652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:34.080300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.080570 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.080841 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] PATH_INFO: `/v3/users/f87a956205624450a75f304dae31a89d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.082084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87a956205624450a75f304dae31a89d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:34.084071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:34.137741 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87a956205624450a75f304dae31a89d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.147701 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87a956205624450a75f304dae31a89d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.148338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87a956205624450a75f304dae31a89d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.163599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87a956205624450a75f304dae31a89d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.164335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06eec319-bc1b-406b-afb8-6abb2756043a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87a956205624450a75f304dae31a89d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.177229 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1405/2839] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:34 2022] DELETE /identity/v3/users/f87a956205624450a75f304dae31a89d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:34.460185 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:34.461698 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:34.475881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997db89-bfb3-433a-9e53-69572154df80 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:34.535920 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:34.536558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.536833 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.537109 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] PATH_INFO: `/v3/projects/d770b6c9957c4627b9314385cd31f474` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.538208 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d770b6c9957c4627b9314385cd31f474)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:34.540166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:34.557212 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:34.591855 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd770b6c9957c4627b9314385cd31f474', 'request_id': 'req-798a55b7-f911-4340-b1c4-df5b133e6d05'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.600220 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd770b6c9957c4627b9314385cd31f474', 'request_id': 'req-798a55b7-f911-4340-b1c4-df5b133e6d05'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.606360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd770b6c9957c4627b9314385cd31f474', 'request_id': 'req-798a55b7-f911-4340-b1c4-df5b133e6d05'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.617001 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] The token cache is being invalidate because project d770b6c9957c4627b9314385cd31f474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:34.617368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.618014 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-798a55b7-f911-4340-b1c4-df5b133e6d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:34.619577 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1435/2840] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:34 2022] DELETE /identity/v3/projects/d770b6c9957c4627b9314385cd31f474 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:34.624702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340df8-fb19-4f81-847e-a0b2b696af73 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.625222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340df8-fb19-4f81-847e-a0b2b696af73 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.625222 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340df8-fb19-4f81-847e-a0b2b696af73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.646938 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8f340df8-fb19-4f81-847e-a0b2b696af73 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:34.648203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f340df8-fb19-4f81-847e-a0b2b696af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:34.712039 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1406/2841] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:34.719608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106997dc-1735-47b9-8170-dd59541b4f55 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.719888 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106997dc-1735-47b9-8170-dd59541b4f55 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.720153 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106997dc-1735-47b9-8170-dd59541b4f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.740346 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-106997dc-1735-47b9-8170-dd59541b4f55 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:34.741532 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-106997dc-1735-47b9-8170-dd59541b4f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:34.749260 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1436/2842] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:34.753484 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f340df8-fb19-4f81-847e-a0b2b696af73 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:34.763687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f340df8-fb19-4f81-847e-a0b2b696af73 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:34.768844 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:34.769444 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.769720 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.770076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.770751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:45:34.771077 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:34.772680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:34.785076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1407/2843] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:45:34.789273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106997dc-1735-47b9-8170-dd59541b4f55 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:34.798534 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106997dc-1735-47b9-8170-dd59541b4f55 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:34.803624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367c76d-5baa-4d3b-9f0c-14dfaa599837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:34.804223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c76d-5baa-4d3b-9f0c-14dfaa599837 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.804495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c76d-5baa-4d3b-9f0c-14dfaa599837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.804754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c76d-5baa-4d3b-9f0c-14dfaa599837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.805390 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367c76d-5baa-4d3b-9f0c-14dfaa599837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:34.807316 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367c76d-5baa-4d3b-9f0c-14dfaa599837 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:34.838115 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1437/2844] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:34 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:34.842683 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:34.853118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27dac3-cbf0-4d97-9a2b-8a8c7845778d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:34.879753 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:34.915430 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a4e51-4fb2-4d16-a423-d823718f7eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:34.916040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a4e51-4fb2-4d16-a423-d823718f7eba admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.916344 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a4e51-4fb2-4d16-a423-d823718f7eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.916622 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a4e51-4fb2-4d16-a423-d823718f7eba admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.917244 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a4e51-4fb2-4d16-a423-d823718f7eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:34.919192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a4e51-4fb2-4d16-a423-d823718f7eba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:34.942280 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1408/2845] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:34 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:34.946846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367c76d-5baa-4d3b-9f0c-14dfaa599837 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:34.956992 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367c76d-5baa-4d3b-9f0c-14dfaa599837 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:34.961575 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78831d-acf5-472a-80f1-c40c000c53fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:34.962235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78831d-acf5-472a-80f1-c40c000c53fd admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.962439 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78831d-acf5-472a-80f1-c40c000c53fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.962706 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78831d-acf5-472a-80f1-c40c000c53fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.963369 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78831d-acf5-472a-80f1-c40c000c53fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:34.964947 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78831d-acf5-472a-80f1-c40c000c53fd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:34.974690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1438/2846] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:34.978467 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a4e51-4fb2-4d16-a423-d823718f7eba admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:34.988907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a4e51-4fb2-4d16-a423-d823718f7eba admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:34.993719 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcc56d-058a-4489-881b-9e4043f8516b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:34.994306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcc56d-058a-4489-881b-9e4043f8516b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:34.994579 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcc56d-058a-4489-881b-9e4043f8516b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:34.994879 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcc56d-058a-4489-881b-9e4043f8516b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:34.995527 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcc56d-058a-4489-881b-9e4043f8516b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:34.997120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcc56d-058a-4489-881b-9e4043f8516b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:35.009047 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1409/2847] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:35.014140 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78831d-acf5-472a-80f1-c40c000c53fd admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:35.030155 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78831d-acf5-472a-80f1-c40c000c53fd admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:35.036911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:35.037707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:35.038056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:35.038400 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] PATH_INFO: `/v3/projects/4877febbcf7543f9b4ba73cf56a27b47/users/0b380f1abbc748f88b76d3328452424d/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:35.060734 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4877febbcf7543f9b4ba73cf56a27b47, user_id=0b380f1abbc748f88b76d3328452424d, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:35.063790 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:35.079417 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1439/2848] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:35 2022] PUT /identity/v3/projects/4877febbcf7543f9b4ba73cf56a27b47/users/0b380f1abbc748f88b76d3328452424d/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:35.084892 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9dc80-a2cd-46ab-93bd-ce2fa297983b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:35.085180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9dc80-a2cd-46ab-93bd-ce2fa297983b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:35.085454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9dc80-a2cd-46ab-93bd-ce2fa297983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:35.105692 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cae9dc80-a2cd-46ab-93bd-ce2fa297983b None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:35.106980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cae9dc80-a2cd-46ab-93bd-ce2fa297983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:35.178085 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1410/2849] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:35.191162 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:35.193108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:35.263182 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60437de-990e-4c43-a2a3-f8582448dae8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:35.268000 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268ea6e-3559-463f-b036-ef6ae057dfc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:35.268646 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268ea6e-3559-463f-b036-ef6ae057dfc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:35.268926 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268ea6e-3559-463f-b036-ef6ae057dfc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:35.269200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268ea6e-3559-463f-b036-ef6ae057dfc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:35.278655 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268ea6e-3559-463f-b036-ef6ae057dfc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:35.280733 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268ea6e-3559-463f-b036-ef6ae057dfc5 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:35.290246 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1440/2850] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:35.928335 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ef37-7628-4270-8d20-77587f874a56 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:35.928847 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ef37-7628-4270-8d20-77587f874a56 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:35.928975 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ef37-7628-4270-8d20-77587f874a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:35.959861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cdc7ef37-7628-4270-8d20-77587f874a56 None None] MFA Rules not processed for user `0b380f1abbc748f88b76d3328452424d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:35.961066 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cdc7ef37-7628-4270-8d20-77587f874a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:36.029084 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1411/2851] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:35 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:36.033778 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268ea6e-3559-463f-b036-ef6ae057dfc5 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:36.053725 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268ea6e-3559-463f-b036-ef6ae057dfc5 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:36.059364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba78a8-b177-44c3-bc75-f6eb41bcdabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:36.060192 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba78a8-b177-44c3-bc75-f6eb41bcdabe admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:36.060690 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba78a8-b177-44c3-bc75-f6eb41bcdabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:36.061371 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba78a8-b177-44c3-bc75-f6eb41bcdabe admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:36.062071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba78a8-b177-44c3-bc75-f6eb41bcdabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:36.064199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba78a8-b177-44c3-bc75-f6eb41bcdabe admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:36.097384 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1441/2852] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:36 2022] POST /identity/v3/projects => generated 403 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:36.098930 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ef37-7628-4270-8d20-77587f874a56 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:36.110060 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ef37-7628-4270-8d20-77587f874a56 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:36.142562 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:36.166812 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6daac-0e60-4f45-bf25-69e41d0ffbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:36.167425 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6daac-0e60-4f45-bf25-69e41d0ffbba admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:36.167702 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6daac-0e60-4f45-bf25-69e41d0ffbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:36.167971 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6daac-0e60-4f45-bf25-69e41d0ffbba admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:36.168605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6daac-0e60-4f45-bf25-69e41d0ffbba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:36.170677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6daac-0e60-4f45-bf25-69e41d0ffbba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:36.193377 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1412/2853] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:36 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:45:36.197665 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba78a8-b177-44c3-bc75-f6eb41bcdabe admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:36.208804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba78a8-b177-44c3-bc75-f6eb41bcdabe admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:36.214707 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470250e-3679-420c-add2-afa50d4f3b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:36.215347 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470250e-3679-420c-add2-afa50d4f3b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:36.215631 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470250e-3679-420c-add2-afa50d4f3b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:36.215923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470250e-3679-420c-add2-afa50d4f3b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:36.216598 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470250e-3679-420c-add2-afa50d4f3b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:36.218221 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470250e-3679-420c-add2-afa50d4f3b14 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:36.229429 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1442/2854] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:36.233589 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6daac-0e60-4f45-bf25-69e41d0ffbba admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:36.243873 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6daac-0e60-4f45-bf25-69e41d0ffbba admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:36.248806 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650af8-2f64-4bc7-82ca-10d4064922a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:36.249445 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650af8-2f64-4bc7-82ca-10d4064922a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:36.249716 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650af8-2f64-4bc7-82ca-10d4064922a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:36.250011 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650af8-2f64-4bc7-82ca-10d4064922a8 admin admin] PATH_INFO: `/v3/projects/94823d218f6f4c89b910b81104171e72/users/a49a2d91f63d4a66b74da67471c6610e/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:36.265723 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7650af8-2f64-4bc7-82ca-10d4064922a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94823d218f6f4c89b910b81104171e72, user_id=a49a2d91f63d4a66b74da67471c6610e, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:36.268408 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7650af8-2f64-4bc7-82ca-10d4064922a8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:36.283225 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1413/2855] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:36 2022] PUT /identity/v3/projects/94823d218f6f4c89b910b81104171e72/users/a49a2d91f63d4a66b74da67471c6610e/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:36.288693 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470250e-3679-420c-add2-afa50d4f3b14 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:36.299510 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470250e-3679-420c-add2-afa50d4f3b14 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:36.308149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:36.360454 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7fb3e-3aef-4c84-9cc4-6c9cf7e9df99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:36.361056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7fb3e-3aef-4c84-9cc4-6c9cf7e9df99 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:36.361335 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7fb3e-3aef-4c84-9cc4-6c9cf7e9df99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:36.361602 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7fb3e-3aef-4c84-9cc4-6c9cf7e9df99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:36.362239 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7fb3e-3aef-4c84-9cc4-6c9cf7e9df99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:36.363873 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7fb3e-3aef-4c84-9cc4-6c9cf7e9df99 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:36.373667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1443/2856] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:45:36.378299 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650af8-2f64-4bc7-82ca-10d4064922a8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:36.389034 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650af8-2f64-4bc7-82ca-10d4064922a8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:36.394148 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf7bc9-19d4-49e3-bedc-dc40799c5db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:36.394944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7bc9-19d4-49e3-bedc-dc40799c5db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:36.395221 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7bc9-19d4-49e3-bedc-dc40799c5db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:36.395491 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7bc9-19d4-49e3-bedc-dc40799c5db4 admin admin] PATH_INFO: `/v3/projects/94823d218f6f4c89b910b81104171e72/users/a49a2d91f63d4a66b74da67471c6610e/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:36.397540 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf7bc9-19d4-49e3-bedc-dc40799c5db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94823d218f6f4c89b910b81104171e72, user_id=a49a2d91f63d4a66b74da67471c6610e, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:36.400235 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf7bc9-19d4-49e3-bedc-dc40799c5db4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:36.410496 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1414/2857] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:36 2022] PUT /identity/v3/projects/94823d218f6f4c89b910b81104171e72/users/a49a2d91f63d4a66b74da67471c6610e/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:37.325558 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2e1c7-1c2e-427b-8b5a-18cf74fffba4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:37.326261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2e1c7-1c2e-427b-8b5a-18cf74fffba4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:37.326261 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2e1c7-1c2e-427b-8b5a-18cf74fffba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:37.354708 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0ac2e1c7-1c2e-427b-8b5a-18cf74fffba4 None None] MFA Rules not processed for user `a49a2d91f63d4a66b74da67471c6610e`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:37.355958 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0ac2e1c7-1c2e-427b-8b5a-18cf74fffba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:37.426858 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1444/2858] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:37 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:38.494096 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf7bc9-19d4-49e3-bedc-dc40799c5db4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:38.568687 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf7bc9-19d4-49e3-bedc-dc40799c5db4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:38.574098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:38.574746 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:38.575022 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:38.575293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] PATH_INFO: `/v3/users/0b380f1abbc748f88b76d3328452424d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:38.576481 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b380f1abbc748f88b76d3328452424d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:38.578393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:38.626651 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b380f1abbc748f88b76d3328452424d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:38.635568 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b380f1abbc748f88b76d3328452424d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:38.636191 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b380f1abbc748f88b76d3328452424d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:38.647778 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b380f1abbc748f88b76d3328452424d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:38.648389 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b380f1abbc748f88b76d3328452424d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:38.661516 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1415/2859] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:38 2022] DELETE /identity/v3/users/0b380f1abbc748f88b76d3328452424d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:38.837787 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2e1c7-1c2e-427b-8b5a-18cf74fffba4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:38.851460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2e1c7-1c2e-427b-8b5a-18cf74fffba4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:38.917526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:38.918071 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:38.918349 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:38.918921 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] PATH_INFO: `/v3/projects/4877febbcf7543f9b4ba73cf56a27b47` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:38.920524 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4877febbcf7543f9b4ba73cf56a27b47)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:38.922530 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:38.961692 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:38.972273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4877febbcf7543f9b4ba73cf56a27b47', 'request_id': 'req-c1789f95-6df7-4864-a604-88c9e670a0a2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:38.981348 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4877febbcf7543f9b4ba73cf56a27b47', 'request_id': 'req-c1789f95-6df7-4864-a604-88c9e670a0a2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:38.988053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4877febbcf7543f9b4ba73cf56a27b47', 'request_id': 'req-c1789f95-6df7-4864-a604-88c9e670a0a2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:38.999446 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] The token cache is being invalidate because project 4877febbcf7543f9b4ba73cf56a27b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:38.999732 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.000465 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.002038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1445/2860] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:38 2022] DELETE /identity/v3/projects/4877febbcf7543f9b4ba73cf56a27b47 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:39.006343 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:39.007817 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:39.021094 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd1c06-7f04-430a-9cb8-cd22d4f70057 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:39.026173 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:39.086318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:39.087058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.087338 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.087593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] PATH_INFO: `/v3/users/a49a2d91f63d4a66b74da67471c6610e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.088726 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49a2d91f63d4a66b74da67471c6610e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:39.091214 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:39.145786 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49a2d91f63d4a66b74da67471c6610e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.154882 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49a2d91f63d4a66b74da67471c6610e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.155577 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49a2d91f63d4a66b74da67471c6610e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.169192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49a2d91f63d4a66b74da67471c6610e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.169499 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ef6c70e-7d5c-48df-a7eb-2f8bc1279c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49a2d91f63d4a66b74da67471c6610e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.181669 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1416/2861] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:39 2022] DELETE /identity/v3/users/a49a2d91f63d4a66b74da67471c6610e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:39.365034 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:39.366616 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:39.379477 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789f95-6df7-4864-a604-88c9e670a0a2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:39.440762 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:39.441329 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.441608 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.441882 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] PATH_INFO: `/v3/projects/94823d218f6f4c89b910b81104171e72` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.443158 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94823d218f6f4c89b910b81104171e72)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:39.445060 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:39.495161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94823d218f6f4c89b910b81104171e72', 'request_id': 'req-23a836a4-d9fc-446d-8847-b5d4feebf141'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.504825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94823d218f6f4c89b910b81104171e72', 'request_id': 'req-23a836a4-d9fc-446d-8847-b5d4feebf141'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.511149 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94823d218f6f4c89b910b81104171e72', 'request_id': 'req-23a836a4-d9fc-446d-8847-b5d4feebf141'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.523356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] The token cache is being invalidate because project 94823d218f6f4c89b910b81104171e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:39.523642 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.524352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-23a836a4-d9fc-446d-8847-b5d4feebf141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:39.525822 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1446/2862] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:39 2022] DELETE /identity/v3/projects/94823d218f6f4c89b910b81104171e72 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:39.532003 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504398f-fafb-44bd-8148-2f4775f74552 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.532525 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504398f-fafb-44bd-8148-2f4775f74552 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.532636 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504398f-fafb-44bd-8148-2f4775f74552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.561240 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d504398f-fafb-44bd-8148-2f4775f74552 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:39.562435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d504398f-fafb-44bd-8148-2f4775f74552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:39.622510 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1417/2863] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:39.630643 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44288-c7f2-450c-834a-a5415398688e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.630956 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44288-c7f2-450c-834a-a5415398688e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.631225 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace44288-c7f2-450c-834a-a5415398688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.651615 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ace44288-c7f2-450c-834a-a5415398688e None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:39.652968 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ace44288-c7f2-450c-834a-a5415398688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:39.662574 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1447/2864] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:39.667117 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504398f-fafb-44bd-8148-2f4775f74552 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:39.677695 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504398f-fafb-44bd-8148-2f4775f74552 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:39.683793 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:39.684472 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.684815 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.685449 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.686098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:45:39.686402 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:39.688035 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:39.701578 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1418/2865] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:45:39.705319 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace44288-c7f2-450c-834a-a5415398688e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:39.715008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace44288-c7f2-450c-834a-a5415398688e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:39.720668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2aad6-e009-45d6-9bd3-ba889b49750b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:39.721326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2aad6-e009-45d6-9bd3-ba889b49750b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.721604 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2aad6-e009-45d6-9bd3-ba889b49750b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.721867 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2aad6-e009-45d6-9bd3-ba889b49750b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.722483 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2aad6-e009-45d6-9bd3-ba889b49750b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:39.724460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2aad6-e009-45d6-9bd3-ba889b49750b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:39.753892 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1448/2866] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:39 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:39.758275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:39.768435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c645744-25bf-4ab1-9e73-a7d188dec9dd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:39.780460 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:39.830375 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85a666-492b-4c1e-9b15-67b969096e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:39.830999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85a666-492b-4c1e-9b15-67b969096e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.831271 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85a666-492b-4c1e-9b15-67b969096e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.831537 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85a666-492b-4c1e-9b15-67b969096e60 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.832194 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85a666-492b-4c1e-9b15-67b969096e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:39.834098 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85a666-492b-4c1e-9b15-67b969096e60 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:39.856253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1419/2867] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:39 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:45:39.862828 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2aad6-e009-45d6-9bd3-ba889b49750b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:39.879184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2aad6-e009-45d6-9bd3-ba889b49750b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:39.887286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bc3ae-ef3e-4e07-95ce-2274f108f0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:39.887899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc3ae-ef3e-4e07-95ce-2274f108f0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.888256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc3ae-ef3e-4e07-95ce-2274f108f0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.888416 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc3ae-ef3e-4e07-95ce-2274f108f0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.889057 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bc3ae-ef3e-4e07-95ce-2274f108f0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:39.890663 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bc3ae-ef3e-4e07-95ce-2274f108f0ce admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:39.901341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1449/2868] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:39.905573 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85a666-492b-4c1e-9b15-67b969096e60 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:39.916366 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85a666-492b-4c1e-9b15-67b969096e60 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:39.921176 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a1a54-79e7-430c-8cb4-bd9c50c33ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:39.921771 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a1a54-79e7-430c-8cb4-bd9c50c33ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.922049 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a1a54-79e7-430c-8cb4-bd9c50c33ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.922326 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a1a54-79e7-430c-8cb4-bd9c50c33ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.923006 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a1a54-79e7-430c-8cb4-bd9c50c33ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:39.924605 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a1a54-79e7-430c-8cb4-bd9c50c33ea5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:39.937264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1420/2869] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:45:39.941471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bc3ae-ef3e-4e07-95ce-2274f108f0ce admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:39.952266 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bc3ae-ef3e-4e07-95ce-2274f108f0ce admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:39.957398 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed30e3a-7378-4dd3-b39c-12de5bfa0ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:39.958038 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed30e3a-7378-4dd3-b39c-12de5bfa0ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:39.958323 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed30e3a-7378-4dd3-b39c-12de5bfa0ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:39.958584 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed30e3a-7378-4dd3-b39c-12de5bfa0ab9 admin admin] PATH_INFO: `/v3/projects/d4cd06805fa84925929181432eb3711b/users/d53b257ca98a475da499c656686899b2/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:39.969923 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:39.974362 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed30e3a-7378-4dd3-b39c-12de5bfa0ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cd06805fa84925929181432eb3711b, user_id=d53b257ca98a475da499c656686899b2, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:39.977488 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed30e3a-7378-4dd3-b39c-12de5bfa0ab9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:39.993040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1450/2870] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:39 2022] PUT /identity/v3/projects/d4cd06805fa84925929181432eb3711b/users/d53b257ca98a475da499c656686899b2/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:40.004599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4642dd-c28f-44d9-ac14-8e040d65f524 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.004878 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4642dd-c28f-44d9-ac14-8e040d65f524 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.005141 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4642dd-c28f-44d9-ac14-8e040d65f524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.032823 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ef4642dd-c28f-44d9-ac14-8e040d65f524 None None] MFA Rules not processed for user `d53b257ca98a475da499c656686899b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:40.034025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ef4642dd-c28f-44d9-ac14-8e040d65f524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:40.098432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1421/2871] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:40.103135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed30e3a-7378-4dd3-b39c-12de5bfa0ab9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:40.164540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed30e3a-7378-4dd3-b39c-12de5bfa0ab9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:40.169645 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80c7b1-5908-4dbd-9f4d-4135999e1d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:40.170240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c7b1-5908-4dbd-9f4d-4135999e1d77 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.170514 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c7b1-5908-4dbd-9f4d-4135999e1d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.170808 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c7b1-5908-4dbd-9f4d-4135999e1d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.171421 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80c7b1-5908-4dbd-9f4d-4135999e1d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:40.173300 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80c7b1-5908-4dbd-9f4d-4135999e1d77 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:40.214775 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1451/2872] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:40 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:45:40.221418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4642dd-c28f-44d9-ac14-8e040d65f524 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:40.237158 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4642dd-c28f-44d9-ac14-8e040d65f524 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:40.294831 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7c62-7495-4001-ae43-8f6ae1634350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:40.295432 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7c62-7495-4001-ae43-8f6ae1634350 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.295708 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7c62-7495-4001-ae43-8f6ae1634350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.295970 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7c62-7495-4001-ae43-8f6ae1634350 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.296591 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed7c62-7495-4001-ae43-8f6ae1634350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:40.298488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed7c62-7495-4001-ae43-8f6ae1634350 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:40.321057 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1422/2873] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:40 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:40.325205 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80c7b1-5908-4dbd-9f4d-4135999e1d77 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:40.335628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80c7b1-5908-4dbd-9f4d-4135999e1d77 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:40.340283 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e581f-2fa7-4d5e-843c-880591c53b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:40.340881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e581f-2fa7-4d5e-843c-880591c53b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.341152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e581f-2fa7-4d5e-843c-880591c53b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.341415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e581f-2fa7-4d5e-843c-880591c53b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.342072 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e581f-2fa7-4d5e-843c-880591c53b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:40.343716 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e581f-2fa7-4d5e-843c-880591c53b7b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:40.354248 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1452/2874] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:40.360614 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7c62-7495-4001-ae43-8f6ae1634350 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:40.376777 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7c62-7495-4001-ae43-8f6ae1634350 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:40.385580 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bd3d3-d1bd-49ba-a75c-f19aeef134ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:40.386511 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bd3d3-d1bd-49ba-a75c-f19aeef134ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.386835 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bd3d3-d1bd-49ba-a75c-f19aeef134ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.387182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bd3d3-d1bd-49ba-a75c-f19aeef134ea admin admin] PATH_INFO: `/v3/projects/8e07c03bf578461a83e08fdb9b0700ea/users/0ae15172f681417eb6ca34de11cf98bb/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.399508 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:40.404658 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bd3d3-d1bd-49ba-a75c-f19aeef134ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e07c03bf578461a83e08fdb9b0700ea, user_id=0ae15172f681417eb6ca34de11cf98bb, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:40.408044 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bd3d3-d1bd-49ba-a75c-f19aeef134ea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:40.423564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1423/2875] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:40 2022] PUT /identity/v3/projects/8e07c03bf578461a83e08fdb9b0700ea/users/0ae15172f681417eb6ca34de11cf98bb/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:40.427540 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e581f-2fa7-4d5e-843c-880591c53b7b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:40.438404 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e581f-2fa7-4d5e-843c-880591c53b7b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:40.500687 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5ebcc-fa7a-421c-93fe-9f00bda65edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:40.501218 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5ebcc-fa7a-421c-93fe-9f00bda65edd admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.501487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5ebcc-fa7a-421c-93fe-9f00bda65edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.501755 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5ebcc-fa7a-421c-93fe-9f00bda65edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.502387 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5ebcc-fa7a-421c-93fe-9f00bda65edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:40.504033 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5ebcc-fa7a-421c-93fe-9f00bda65edd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:40.513938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1453/2876] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:40.519300 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bd3d3-d1bd-49ba-a75c-f19aeef134ea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:40.531092 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bd3d3-d1bd-49ba-a75c-f19aeef134ea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:40.537070 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:40.537707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.537992 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.538260 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] PATH_INFO: `/v3/projects/8e07c03bf578461a83e08fdb9b0700ea/users/0ae15172f681417eb6ca34de11cf98bb/roles/241e99d365564c0fa95cb712f4004785` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.540543 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e07c03bf578461a83e08fdb9b0700ea, user_id=0ae15172f681417eb6ca34de11cf98bb, role_id=241e99d365564c0fa95cb712f4004785)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:40.543208 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:40.554264 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1424/2877] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:40 2022] PUT /identity/v3/projects/8e07c03bf578461a83e08fdb9b0700ea/users/0ae15172f681417eb6ca34de11cf98bb/roles/241e99d365564c0fa95cb712f4004785 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:40.566563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5d11-3245-453d-9c4b-8f4e5747fa7d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.566938 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5d11-3245-453d-9c4b-8f4e5747fa7d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.567288 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5d11-3245-453d-9c4b-8f4e5747fa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.604268 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-64af5d11-3245-453d-9c4b-8f4e5747fa7d None None] MFA Rules not processed for user `0ae15172f681417eb6ca34de11cf98bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:40.605458 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-64af5d11-3245-453d-9c4b-8f4e5747fa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:40.651850 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:40.674549 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1454/2878] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:40.686400 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:40.688258 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:40.748120 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c26a12-abc9-4810-b616-305c20916d74 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:40.752131 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce367852-f477-486e-b343-5a3b3e03ac65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:40.752653 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce367852-f477-486e-b343-5a3b3e03ac65 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.752922 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce367852-f477-486e-b343-5a3b3e03ac65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.753186 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce367852-f477-486e-b343-5a3b3e03ac65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.762641 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce367852-f477-486e-b343-5a3b3e03ac65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:40.764125 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce367852-f477-486e-b343-5a3b3e03ac65 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:40.773652 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1425/2879] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:40.854326 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af5d11-3245-453d-9c4b-8f4e5747fa7d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:40.865274 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af5d11-3245-453d-9c4b-8f4e5747fa7d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:40.869361 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:40.869962 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:40.870235 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:40.870502 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:40.933969 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:40.935964 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:40.945632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1455/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:50.769442 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=35,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:50.941255 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=35 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:53.654521 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245cf13-1886-4a0e-aa7c-b6b851cd153a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:53.655078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245cf13-1886-4a0e-aa7c-b6b851cd153a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:53.655078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245cf13-1886-4a0e-aa7c-b6b851cd153a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:53.674996 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5245cf13-1886-4a0e-aa7c-b6b851cd153a None None] MFA Rules not processed for user `c11d44667e8e4db5bccccaee16d89a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:53.676199 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5245cf13-1886-4a0e-aa7c-b6b851cd153a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:53.742312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1426/2881] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:54.081425 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:54.086509 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:54.145011 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0a5a-459b-459e-be34-97a0a8b5d8b1 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:54.148961 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec42a51-c58c-4d70-badc-b919ad3f8b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:54.149593 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec42a51-c58c-4d70-badc-b919ad3f8b9f service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:54.149866 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec42a51-c58c-4d70-badc-b919ad3f8b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:54.150765 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec42a51-c58c-4d70-badc-b919ad3f8b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:54.153538 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5ec42a51-c58c-4d70-badc-b919ad3f8b9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:54.165135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec42a51-c58c-4d70-badc-b919ad3f8b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:54.166580 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec42a51-c58c-4d70-badc-b919ad3f8b9f service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:54.175397 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1456/2882] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:54.181488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245cf13-1886-4a0e-aa7c-b6b851cd153a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:54.247725 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245cf13-1886-4a0e-aa7c-b6b851cd153a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:54.254016 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:54.254707 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:54.255021 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:54.255291 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] PATH_INFO: `/v3/users/d53b257ca98a475da499c656686899b2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:54.256452 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53b257ca98a475da499c656686899b2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:54.258418 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:54.306560 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53b257ca98a475da499c656686899b2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.315327 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53b257ca98a475da499c656686899b2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.315928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53b257ca98a475da499c656686899b2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.328711 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53b257ca98a475da499c656686899b2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.329365 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53b257ca98a475da499c656686899b2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.342572 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1427/2883] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:54 2022] DELETE /identity/v3/users/d53b257ca98a475da499c656686899b2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:54.347363 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c580-55dc-46f3-9eb9-df1bf63b7c78 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:54.347657 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c580-55dc-46f3-9eb9-df1bf63b7c78 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:54.347995 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c580-55dc-46f3-9eb9-df1bf63b7c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:54.367803 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2b93c580-55dc-46f3-9eb9-df1bf63b7c78 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:54.369022 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2b93c580-55dc-46f3-9eb9-df1bf63b7c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:54.428763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1457/2884] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:54.440164 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:54.441593 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:54.470770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:54.502933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30857f7d-44dc-43ee-b616-f6b7491ce49e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:54.508289 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c511cfe-965d-4d26-8461-8771b032bf49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:54.508900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c511cfe-965d-4d26-8461-8771b032bf49 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:54.509171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c511cfe-965d-4d26-8461-8771b032bf49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:54.509434 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c511cfe-965d-4d26-8461-8771b032bf49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:54.519869 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c511cfe-965d-4d26-8461-8771b032bf49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:54.521876 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c511cfe-965d-4d26-8461-8771b032bf49 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:54.532530 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1428/2885] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:54.817799 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93c580-55dc-46f3-9eb9-df1bf63b7c78 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:54.819632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2b93c580-55dc-46f3-9eb9-df1bf63b7c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:54.834512 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93c580-55dc-46f3-9eb9-df1bf63b7c78 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:54.839746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:54.841104 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:54.842082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:54.842082 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] PATH_INFO: `/v3/projects/d4cd06805fa84925929181432eb3711b` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:54.843413 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4cd06805fa84925929181432eb3711b)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:54.845634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:54.894041 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4cd06805fa84925929181432eb3711b', 'request_id': 'req-c8c2341a-7b35-494b-b309-1be316879a6a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.904576 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4cd06805fa84925929181432eb3711b', 'request_id': 'req-c8c2341a-7b35-494b-b309-1be316879a6a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.911672 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4cd06805fa84925929181432eb3711b', 'request_id': 'req-c8c2341a-7b35-494b-b309-1be316879a6a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.923004 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] The token cache is being invalidate because project d4cd06805fa84925929181432eb3711b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:54.923287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.923918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:54.925410 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1458/2886] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:54 2022] DELETE /identity/v3/projects/d4cd06805fa84925929181432eb3711b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:45:54.929608 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c511cfe-965d-4d26-8461-8771b032bf49 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:54.939846 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c511cfe-965d-4d26-8461-8771b032bf49 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:55.000564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:55.001192 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.001569 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.001840 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] PATH_INFO: `/v3/users/0ae15172f681417eb6ca34de11cf98bb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.003197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae15172f681417eb6ca34de11cf98bb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:55.005146 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:55.051139 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae15172f681417eb6ca34de11cf98bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.060477 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae15172f681417eb6ca34de11cf98bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.061319 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae15172f681417eb6ca34de11cf98bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.075284 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae15172f681417eb6ca34de11cf98bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.076103 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b343f0ee-b7c0-4056-b781-a70cd2d2b3e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae15172f681417eb6ca34de11cf98bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.088751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1429/2887] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:54 2022] DELETE /identity/v3/users/0ae15172f681417eb6ca34de11cf98bb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:55.376675 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:55.378292 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:55.394402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2341a-7b35-494b-b309-1be316879a6a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:55.421054 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:55.453988 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:55.454624 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.454929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.455200 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] PATH_INFO: `/v3/projects/8e07c03bf578461a83e08fdb9b0700ea` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.456306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e07c03bf578461a83e08fdb9b0700ea)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:55.458250 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:55.506619 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e07c03bf578461a83e08fdb9b0700ea', 'request_id': 'req-4472af7e-41a1-4d04-b727-2e9d3b720fcd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.514191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e07c03bf578461a83e08fdb9b0700ea', 'request_id': 'req-4472af7e-41a1-4d04-b727-2e9d3b720fcd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.521721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e07c03bf578461a83e08fdb9b0700ea', 'request_id': 'req-4472af7e-41a1-4d04-b727-2e9d3b720fcd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.534264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] The token cache is being invalidate because project 8e07c03bf578461a83e08fdb9b0700ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:55.534547 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.535350 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4472af7e-41a1-4d04-b727-2e9d3b720fcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:55.536997 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1459/2888] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:55 2022] DELETE /identity/v3/projects/8e07c03bf578461a83e08fdb9b0700ea => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:55.542288 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371478-05e9-4797-b4a6-881ac20e776a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.542630 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371478-05e9-4797-b4a6-881ac20e776a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.542939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371478-05e9-4797-b4a6-881ac20e776a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.564165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a5371478-05e9-4797-b4a6-881ac20e776a None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:55.565361 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a5371478-05e9-4797-b4a6-881ac20e776a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:55.608924 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:55.631781 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1430/2889] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:55.640451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50bccc-9494-4d94-b1df-3be8d45b0b10 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.640824 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50bccc-9494-4d94-b1df-3be8d45b0b10 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.641143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50bccc-9494-4d94-b1df-3be8d45b0b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.662684 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-da50bccc-9494-4d94-b1df-3be8d45b0b10 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:55.663905 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-da50bccc-9494-4d94-b1df-3be8d45b0b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:55.673291 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1460/2890] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:55.677824 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5371478-05e9-4797-b4a6-881ac20e776a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:55.690078 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5371478-05e9-4797-b4a6-881ac20e776a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:55.695617 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:55.696216 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.696488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.696748 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.697556 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:45:55.697900 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:55.700197 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:55.715083 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1431/2891] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:45:55.719620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50bccc-9494-4d94-b1df-3be8d45b0b10 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:55.729710 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50bccc-9494-4d94-b1df-3be8d45b0b10 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:55.735356 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8fae5-7a22-4727-8daf-5473f913631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:55.735996 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8fae5-7a22-4727-8daf-5473f913631b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.736240 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8fae5-7a22-4727-8daf-5473f913631b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.736548 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8fae5-7a22-4727-8daf-5473f913631b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.737194 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8fae5-7a22-4727-8daf-5473f913631b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:55.739257 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8fae5-7a22-4727-8daf-5473f913631b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:55.770152 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1461/2892] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:55 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:55.774574 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:55.785519 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbf830-be2a-4a80-880e-efa29eaf4436 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:55.866805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683b889-0be0-4744-8941-c55283afbd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:55.867623 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683b889-0be0-4744-8941-c55283afbd28 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.867956 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683b889-0be0-4744-8941-c55283afbd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.868315 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683b889-0be0-4744-8941-c55283afbd28 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.869046 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683b889-0be0-4744-8941-c55283afbd28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:55.871331 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683b889-0be0-4744-8941-c55283afbd28 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:55.899861 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1432/2893] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:55 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:45:55.901929 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8fae5-7a22-4727-8daf-5473f913631b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:55.913492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8fae5-7a22-4727-8daf-5473f913631b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:55.919533 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996b9ae-15f0-44c9-9f23-d78c93aa87ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:55.920143 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996b9ae-15f0-44c9-9f23-d78c93aa87ac admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.920415 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996b9ae-15f0-44c9-9f23-d78c93aa87ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.920728 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996b9ae-15f0-44c9-9f23-d78c93aa87ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.921378 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996b9ae-15f0-44c9-9f23-d78c93aa87ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:55.923058 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996b9ae-15f0-44c9-9f23-d78c93aa87ac admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:55.933893 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1462/2894] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:55.938088 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683b889-0be0-4744-8941-c55283afbd28 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:55.948678 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683b889-0be0-4744-8941-c55283afbd28 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:55.953775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba26d4-b5ea-470a-ba50-aa6e1dad0159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:55.954378 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba26d4-b5ea-470a-ba50-aa6e1dad0159 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.954703 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba26d4-b5ea-470a-ba50-aa6e1dad0159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.954999 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba26d4-b5ea-470a-ba50-aa6e1dad0159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:55.955646 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba26d4-b5ea-470a-ba50-aa6e1dad0159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:55.957268 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba26d4-b5ea-470a-ba50-aa6e1dad0159 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:55.968212 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1433/2895] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:45:55.972289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996b9ae-15f0-44c9-9f23-d78c93aa87ac admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:55.982823 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996b9ae-15f0-44c9-9f23-d78c93aa87ac admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:55.987836 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:55.988487 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:55.988791 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:55.989065 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] PATH_INFO: `/v3/projects/b7dc1b33911648abb965c6a881ecd8e5/users/fc3fb763ff364f37877664c78edec778/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:56.004793 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7dc1b33911648abb965c6a881ecd8e5, user_id=fc3fb763ff364f37877664c78edec778, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:56.007669 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:56.023825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1463/2896] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:55 2022] PUT /identity/v3/projects/b7dc1b33911648abb965c6a881ecd8e5/users/fc3fb763ff364f37877664c78edec778/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:56.035312 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3b04-f194-4d99-8c54-a6603fc71784 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:56.035740 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3b04-f194-4d99-8c54-a6603fc71784 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:56.035857 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3b04-f194-4d99-8c54-a6603fc71784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:56.069500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-64aa3b04-f194-4d99-8c54-a6603fc71784 None None] MFA Rules not processed for user `fc3fb763ff364f37877664c78edec778`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:56.071310 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-64aa3b04-f194-4d99-8c54-a6603fc71784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:56.140980 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1434/2897] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:45:56.152906 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:56.154842 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:56.212517 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf6935-8628-438a-8a00-27775f420964 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:56.216364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb83c4-405b-4be1-bf7b-f780df4a15b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:56.216981 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb83c4-405b-4be1-bf7b-f780df4a15b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:56.217258 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb83c4-405b-4be1-bf7b-f780df4a15b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:56.217525 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb83c4-405b-4be1-bf7b-f780df4a15b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:56.223918 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:56.226381 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb83c4-405b-4be1-bf7b-f780df4a15b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:56.228610 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb83c4-405b-4be1-bf7b-f780df4a15b0 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:56.239600 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1464/2898] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:56.259908 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa3b04-f194-4d99-8c54-a6603fc71784 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:56.309789 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:56.328788 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa3b04-f194-4d99-8c54-a6603fc71784 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:56.333350 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:56.333986 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:56.334263 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:56.334552 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] PATH_INFO: `/v3/users/fc3fb763ff364f37877664c78edec778` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:56.335765 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3fb763ff364f37877664c78edec778)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:56.337677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:56.391058 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3fb763ff364f37877664c78edec778'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:56.401040 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3fb763ff364f37877664c78edec778'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:56.401825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3fb763ff364f37877664c78edec778'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:56.414820 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3fb763ff364f37877664c78edec778'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:56.415534 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3fb763ff364f37877664c78edec778'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:56.429403 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1435/2899] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:45:56 2022] DELETE /identity/v3/users/fc3fb763ff364f37877664c78edec778 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:56.434457 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41cad4-f8d8-4ec6-8540-36a91fd16098 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:56.435053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41cad4-f8d8-4ec6-8540-36a91fd16098 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:56.435053 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41cad4-f8d8-4ec6-8540-36a91fd16098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:56.454460 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1e41cad4-f8d8-4ec6-8540-36a91fd16098 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:56.455967 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1e41cad4-f8d8-4ec6-8540-36a91fd16098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:56.548287 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1465/2900] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:56.558933 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:56.560376 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:56.652979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ab31b-2c27-4f32-874a-43c056d71267 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:56.658601 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3bd7b-7fd2-41dd-9743-1edc583ca94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:56.659266 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3bd7b-7fd2-41dd-9743-1edc583ca94d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:56.659536 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3bd7b-7fd2-41dd-9743-1edc583ca94d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:56.659969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3bd7b-7fd2-41dd-9743-1edc583ca94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:56.672454 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3bd7b-7fd2-41dd-9743-1edc583ca94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:56.674936 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3bd7b-7fd2-41dd-9743-1edc583ca94d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:56.689542 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1436/2901] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:45:57.113189 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41cad4-f8d8-4ec6-8540-36a91fd16098 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.115135 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1e41cad4-f8d8-4ec6-8540-36a91fd16098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:57.129915 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41cad4-f8d8-4ec6-8540-36a91fd16098 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.143802 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.144463 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.144720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.144977 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] PATH_INFO: `/v3/projects/b7dc1b33911648abb965c6a881ecd8e5` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.146184 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7dc1b33911648abb965c6a881ecd8e5)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.148408 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.204654 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7dc1b33911648abb965c6a881ecd8e5', 'request_id': 'req-44b2af78-4325-4e22-a23f-94cab614149d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:57.218067 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7dc1b33911648abb965c6a881ecd8e5', 'request_id': 'req-44b2af78-4325-4e22-a23f-94cab614149d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:57.226286 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7dc1b33911648abb965c6a881ecd8e5', 'request_id': 'req-44b2af78-4325-4e22-a23f-94cab614149d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:57.237768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] The token cache is being invalidate because project b7dc1b33911648abb965c6a881ecd8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:45:57.238056 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:57.238743 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-44b2af78-4325-4e22-a23f-94cab614149d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:45:57.240238 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1466/2902] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:45:57 2022] DELETE /identity/v3/projects/b7dc1b33911648abb965c6a881ecd8e5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:45:57.245787 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a652-f2d5-4912-8d2a-80cf94ea926d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.246275 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a652-f2d5-4912-8d2a-80cf94ea926d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.246557 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a652-f2d5-4912-8d2a-80cf94ea926d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.266830 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4f80a652-f2d5-4912-8d2a-80cf94ea926d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:57.268172 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4f80a652-f2d5-4912-8d2a-80cf94ea926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:57.337804 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1437/2903] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:57.346023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b20e6-0693-43e4-af44-f35746934dc7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.346305 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b20e6-0693-43e4-af44-f35746934dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.346572 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b20e6-0693-43e4-af44-f35746934dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.354928 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80a652-f2d5-4912-8d2a-80cf94ea926d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.356528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4f80a652-f2d5-4912-8d2a-80cf94ea926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:57.366246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:57.375701 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4c8b20e6-0693-43e4-af44-f35746934dc7 None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:57.376911 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4c8b20e6-0693-43e4-af44-f35746934dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:57.385472 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1467/2904] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:57.392302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b20e6-0693-43e4-af44-f35746934dc7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.407179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b20e6-0693-43e4-af44-f35746934dc7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.412568 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.413086 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.413364 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.413634 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.415003 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:45:57.415330 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.417432 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.434622 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1468/2905] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Aug 31 04:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:45:57.435668 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.448857 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02da51a-5894-4f5b-810a-21826d344444 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.454301 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59568a9f-78eb-4d78-b7e7-ffa68efee2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.454547 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80a652-f2d5-4912-8d2a-80cf94ea926d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.455694 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59568a9f-78eb-4d78-b7e7-ffa68efee2bc admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.456132 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59568a9f-78eb-4d78-b7e7-ffa68efee2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.456523 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59568a9f-78eb-4d78-b7e7-ffa68efee2bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.457191 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59568a9f-78eb-4d78-b7e7-ffa68efee2bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.459111 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59568a9f-78eb-4d78-b7e7-ffa68efee2bc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.460067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.460674 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.460952 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.461220 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.462976 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:57.475845 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.477421 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.488944 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1438/2906] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:45:57.492963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1469/2907] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Aug 31 04:45:57 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:57.499751 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.511080 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1331a-cbef-4462-ac3c-e436d9b64906 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.579014 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61414eb-acfc-4bd5-b776-c712d8f15d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.579634 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61414eb-acfc-4bd5-b776-c712d8f15d47 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.579912 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61414eb-acfc-4bd5-b776-c712d8f15d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.580180 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61414eb-acfc-4bd5-b776-c712d8f15d47 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.580814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61414eb-acfc-4bd5-b776-c712d8f15d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.582774 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61414eb-acfc-4bd5-b776-c712d8f15d47 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.605018 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1439/2908] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Aug 31 04:45:57 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:45:57.609341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59568a9f-78eb-4d78-b7e7-ffa68efee2bc admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.619861 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59568a9f-78eb-4d78-b7e7-ffa68efee2bc admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.624474 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037a280-8b7a-4a5e-8351-06d4e170c7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.625081 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037a280-8b7a-4a5e-8351-06d4e170c7df admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.625357 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037a280-8b7a-4a5e-8351-06d4e170c7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.625632 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037a280-8b7a-4a5e-8351-06d4e170c7df admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.626297 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037a280-8b7a-4a5e-8351-06d4e170c7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.627943 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037a280-8b7a-4a5e-8351-06d4e170c7df admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.638157 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1470/2909] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:57.641984 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61414eb-acfc-4bd5-b776-c712d8f15d47 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.652210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61414eb-acfc-4bd5-b776-c712d8f15d47 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.657246 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5079a4-6d48-48ba-875a-1f2b586c21b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.657872 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5079a4-6d48-48ba-875a-1f2b586c21b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.658202 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5079a4-6d48-48ba-875a-1f2b586c21b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.658394 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5079a4-6d48-48ba-875a-1f2b586c21b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.659132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5079a4-6d48-48ba-875a-1f2b586c21b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.661677 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5079a4-6d48-48ba-875a-1f2b586c21b9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.672881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1440/2910] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Aug 31 04:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:45:57.677232 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037a280-8b7a-4a5e-8351-06d4e170c7df admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.687846 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037a280-8b7a-4a5e-8351-06d4e170c7df admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.692567 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.693219 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.693493 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.693759 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] PATH_INFO: `/v3/projects/afb8ff9bdd6e4b258ddc19e8b1b7905f/users/fba1df547cf34c218415ba1a3bb9abbf/roles/5bbcda16701c451f9419aefc4671840e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.708569 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb8ff9bdd6e4b258ddc19e8b1b7905f, user_id=fba1df547cf34c218415ba1a3bb9abbf, role_id=5bbcda16701c451f9419aefc4671840e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.711256 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.725884 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1471/2911] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Aug 31 04:45:57 2022] PUT /identity/v3/projects/afb8ff9bdd6e4b258ddc19e8b1b7905f/users/fba1df547cf34c218415ba1a3bb9abbf/roles/5bbcda16701c451f9419aefc4671840e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:45:57.737149 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c6ed-0fd7-4850-8c46-840037f9abeb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.737610 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c6ed-0fd7-4850-8c46-840037f9abeb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.737947 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c6ed-0fd7-4850-8c46-840037f9abeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.766896 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9b04c6ed-0fd7-4850-8c46-840037f9abeb None None] MFA Rules not processed for user `fba1df547cf34c218415ba1a3bb9abbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:45:57.768196 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9b04c6ed-0fd7-4850-8c46-840037f9abeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:57.833118 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:57.839175 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1441/2912] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:45:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:45:57.850084 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:45:57.851881 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:45:57.857341 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:45:57.911771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311d0c3-9db0-4e56-91c8-091611ad6662 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:45:57.915480 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a109dac-6f3f-48ac-98d3-1151284a97cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f3a86b76cff4b48a382888cbac913f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:45:57.916088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a109dac-6f3f-48ac-98d3-1151284a97cd service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:45:57.916360 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a109dac-6f3f-48ac-98d3-1151284a97cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:45:57.916627 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a109dac-6f3f-48ac-98d3-1151284a97cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:45:57.924746 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a109dac-6f3f-48ac-98d3-1151284a97cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:45:57.926170 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a109dac-6f3f-48ac-98d3-1151284a97cd service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:45:57.935409 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1472/2913] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:45:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:46:04.041939 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04c6ed-0fd7-4850-8c46-840037f9abeb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:04.119625 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04c6ed-0fd7-4850-8c46-840037f9abeb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:04.126931 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:04.127550 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:04.127814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:04.128075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:04.130498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:04.193994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:04.195629 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:04.206203 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1442/2914] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:46:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:46:07.930891 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=25 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:46:10.531975 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a109dac-6f3f-48ac-98d3-1151284a97cd service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:10.547721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a109dac-6f3f-48ac-98d3-1151284a97cd service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:10.554763 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292cbad-36f0-4c20-9300-c70fd63eeecb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:10.555471 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292cbad-36f0-4c20-9300-c70fd63eeecb service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:10.555720 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292cbad-36f0-4c20-9300-c70fd63eeecb service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:10.556178 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292cbad-36f0-4c20-9300-c70fd63eeecb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:10.558352 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5292cbad-36f0-4c20-9300-c70fd63eeecb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:10.572492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292cbad-36f0-4c20-9300-c70fd63eeecb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:10.574373 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292cbad-36f0-4c20-9300-c70fd63eeecb service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:10.586750 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1473/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:46:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:46:14.201392 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=57 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:46:15.532182 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:15.597494 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f5acd-af65-456d-ba15-69dd03be7d62 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:15.602306 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:15.602993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:15.603269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:15.603539 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] PATH_INFO: `/v3/users/fba1df547cf34c218415ba1a3bb9abbf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:15.604682 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba1df547cf34c218415ba1a3bb9abbf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:15.606615 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:15.656825 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba1df547cf34c218415ba1a3bb9abbf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:15.666087 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba1df547cf34c218415ba1a3bb9abbf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:15.667285 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba1df547cf34c218415ba1a3bb9abbf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:15.684292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba1df547cf34c218415ba1a3bb9abbf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:15.685076 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba1df547cf34c218415ba1a3bb9abbf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:15.697764 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1443/2916] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:46:15 2022] DELETE /identity/v3/users/fba1df547cf34c218415ba1a3bb9abbf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:46:15.702620 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894253d-42a7-4761-a13d-e31291a71d6d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:15.703145 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894253d-42a7-4761-a13d-e31291a71d6d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:15.703263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894253d-42a7-4761-a13d-e31291a71d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:15.722931 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9894253d-42a7-4761-a13d-e31291a71d6d None None] MFA Rules not processed for user `13d95fead9ed4b918730e2198000e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:46:15.724169 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9894253d-42a7-4761-a13d-e31291a71d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:15.783844 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1474/2917] 10.222.0.42 () {58 vars in 958 bytes} [Wed Aug 31 04:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:46:15.793768 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:15.796599 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:15.862488 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d22e7d-1f61-4bcd-af45-6ae82534448c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:15.866056 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:15.866696 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:15.867004 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:15.867276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:15.876475 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:15.877907 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:15.887167 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1444/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:46:16.172354 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894253d-42a7-4761-a13d-e31291a71d6d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:16.173942 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9894253d-42a7-4761-a13d-e31291a71d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:16.188628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894253d-42a7-4761-a13d-e31291a71d6d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:16.193450 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:16.194088 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:16.194368 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:16.194804 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] PATH_INFO: `/v3/projects/afb8ff9bdd6e4b258ddc19e8b1b7905f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:16.195916 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb8ff9bdd6e4b258ddc19e8b1b7905f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:16.197835 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:16.247273 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb8ff9bdd6e4b258ddc19e8b1b7905f', 'request_id': 'req-cd91f126-6eaf-454c-8845-74612f188263'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:16.255952 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb8ff9bdd6e4b258ddc19e8b1b7905f', 'request_id': 'req-cd91f126-6eaf-454c-8845-74612f188263'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:16.263008 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb8ff9bdd6e4b258ddc19e8b1b7905f', 'request_id': 'req-cd91f126-6eaf-454c-8845-74612f188263'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:16.275211 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] The token cache is being invalidate because project afb8ff9bdd6e4b258ddc19e8b1b7905f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:46:16.275501 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:16.276223 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:16.277771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1475/2919] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:46:16 2022] DELETE /identity/v3/projects/afb8ff9bdd6e4b258ddc19e8b1b7905f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:46:25.883210 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:46:26.271407 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:46:45.146705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:45.148075 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:45.235416 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d18f99-58f2-40b3-b696-8fbdcbcd3103 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:45.318025 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '708ce6cfc5f64d26a418b4339fd883fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:45.319059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:45.319059 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:45.319320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:45.321305 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:45.337795 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:45.339456 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:45.353807 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1445/2920] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:46:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:46:51.916166 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:51.919302 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:52.007585 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91f126-6eaf-454c-8845-74612f188263 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:52.013628 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:52.014263 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:52.014536 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:52.014852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:52.016649 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:52.029771 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:52.031566 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:52.043495 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1476/2921] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:46:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:46:55.347293 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=62 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:46:55.468151 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:55.469782 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:55.551166 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821ff4b-8842-4e23-bbe6-08e10ee0fe13 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:55.556171 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:55.556811 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:55.557270 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:55.557949 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] PATH_INFO: `/v3/users/81e0524ede63477a9875487d155be25e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:55.559179 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e0524ede63477a9875487d155be25e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:55.561105 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:55.617412 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e0524ede63477a9875487d155be25e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.627169 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e0524ede63477a9875487d155be25e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.627849 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e0524ede63477a9875487d155be25e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.641498 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e0524ede63477a9875487d155be25e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.642223 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e0524ede63477a9875487d155be25e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.655775 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1446/2922] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:46:55 2022] DELETE /identity/v3/users/81e0524ede63477a9875487d155be25e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:46:55.840739 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:55.854648 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e3c1c-0bce-4631-b398-13f7f74a8df3 service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:55.912699 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:55.913336 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:55.913612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:55.913885 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] PATH_INFO: `/v3/projects/69713aca5ac144f9bca8348430ab9c75` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:55.915066 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69713aca5ac144f9bca8348430ab9c75)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:55.917023 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:55.963764 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69713aca5ac144f9bca8348430ab9c75', 'request_id': 'req-7b304013-5bdf-4dd8-a3c3-8171de3bc754'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.972451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69713aca5ac144f9bca8348430ab9c75', 'request_id': 'req-7b304013-5bdf-4dd8-a3c3-8171de3bc754'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.979173 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69713aca5ac144f9bca8348430ab9c75', 'request_id': 'req-7b304013-5bdf-4dd8-a3c3-8171de3bc754'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.990503 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] The token cache is being invalidate because project 69713aca5ac144f9bca8348430ab9c75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:46:55.990839 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.991609 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:55.993289 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1477/2923] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:46:55 2022] DELETE /identity/v3/projects/69713aca5ac144f9bca8348430ab9c75 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:46:55.997592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:55.999292 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:56.013486 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f67c2-8d0e-4be2-80c1-5cda458325d2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:56.071356 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:56.071994 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:56.072269 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:56.072730 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] PATH_INFO: `/v3/users/5ea488935db448738c23d196a1a3b6b3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:56.074071 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea488935db448738c23d196a1a3b6b3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:56.076026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:56.097528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:46:56.124665 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea488935db448738c23d196a1a3b6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.132913 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea488935db448738c23d196a1a3b6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.133505 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea488935db448738c23d196a1a3b6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.145233 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea488935db448738c23d196a1a3b6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.145881 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea488935db448738c23d196a1a3b6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.157802 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1447/2924] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:46:55 2022] DELETE /identity/v3/users/5ea488935db448738c23d196a1a3b6b3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:46:56.339077 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:46:56.340625 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:46:56.356909 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b304013-5bdf-4dd8-a3c3-8171de3bc754 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:46:56.373777 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:46:56.421721 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:46:56.422388 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:46:56.422667 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:46:56.422985 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] PATH_INFO: `/v3/projects/ad13a1bcbf964a5fb2eb7cc577f8a8e3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:46:56.424124 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad13a1bcbf964a5fb2eb7cc577f8a8e3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:46:56.426092 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:46:56.473837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad13a1bcbf964a5fb2eb7cc577f8a8e3', 'request_id': 'req-372996b2-632e-47f5-b350-cc43409b9547'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.482306 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad13a1bcbf964a5fb2eb7cc577f8a8e3', 'request_id': 'req-372996b2-632e-47f5-b350-cc43409b9547'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.488451 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad13a1bcbf964a5fb2eb7cc577f8a8e3', 'request_id': 'req-372996b2-632e-47f5-b350-cc43409b9547'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.499264 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] The token cache is being invalidate because project ad13a1bcbf964a5fb2eb7cc577f8a8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:46:56.499546 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.500171 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:46:56.501630 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1478/2925] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:46:56 2022] DELETE /identity/v3/projects/ad13a1bcbf964a5fb2eb7cc577f8a8e3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:47:06.154745 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:47:06.496583 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:47:57.803633 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:47:57.805193 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:47:57.878750 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b83eb4-07e2-41b6-b381-fee7f30a6fb5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:47:57.940805 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:47:57.941435 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:47:57.941705 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:47:57.941989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:47:57.943827 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:47:57.955969 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:47:57.957549 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:47:57.967500 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1448/2926] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:47:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:48:07.563402 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:48:07.564838 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:48:07.621899 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372996b2-632e-47f5-b350-cc43409b9547 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:48:07.625754 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13603c6cd9544f5391256ea600ad0709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:48:07.626359 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:48:07.626633 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:48:07.626935 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:48:07.628825 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:48:07.691641 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:48:07.693073 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:48:07.703089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1479/2927] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:48:07.963320 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=58 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:48:10.603860 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:48:10.605406 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:48:10.625033 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b49f9-46a5-43e2-9c9f-8b00ec7683b6 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:48:10.630582 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:48:10.631253 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:48:10.631564 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:48:10.631680 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:48:10.634393 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:48:10.658038 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:48:10.660067 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:48:10.673165 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1449/2928] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:48:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:48:14.154389 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:48:14.155768 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:48:14.168939 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded87b6-94ab-43a5-8051-f8c97156cdb3 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:48:14.174089 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:48:14.174753 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:48:14.175037 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:48:14.175333 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] PATH_INFO: `/v3/users/c11d44667e8e4db5bccccaee16d89a51` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:48:14.176498 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11d44667e8e4db5bccccaee16d89a51)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:48:14.178431 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:48:14.228470 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11d44667e8e4db5bccccaee16d89a51'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.238383 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11d44667e8e4db5bccccaee16d89a51'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.239100 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11d44667e8e4db5bccccaee16d89a51'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.252231 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11d44667e8e4db5bccccaee16d89a51'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.252945 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11d44667e8e4db5bccccaee16d89a51'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.269062 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1480/2929] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:48:14 2022] DELETE /identity/v3/users/c11d44667e8e4db5bccccaee16d89a51 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:48:14.449868 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:48:14.461528 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ac7b8-638d-4f8d-9529-0ea8c6667dff service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:48:14.519843 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:48:14.520480 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:48:14.520755 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:48:14.521026 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] PATH_INFO: `/v3/projects/312b9335479c4d72b78a20bb2f1b7296` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:48:14.522132 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] RBAC: Authorizing `identity:delete_project(project_id=312b9335479c4d72b78a20bb2f1b7296)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:48:14.524345 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:48:14.574979 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312b9335479c4d72b78a20bb2f1b7296', 'request_id': 'req-7947f040-c87b-4311-be3b-3603c0c4266c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.582032 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312b9335479c4d72b78a20bb2f1b7296', 'request_id': 'req-7947f040-c87b-4311-be3b-3603c0c4266c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.592042 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312b9335479c4d72b78a20bb2f1b7296', 'request_id': 'req-7947f040-c87b-4311-be3b-3603c0c4266c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.603522 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] The token cache is being invalidate because project 312b9335479c4d72b78a20bb2f1b7296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:48:14.603790 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.604461 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.605993 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1450/2930] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:48:14 2022] DELETE /identity/v3/projects/312b9335479c4d72b78a20bb2f1b7296 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:48:14.610325 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:48:14.611837 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:48:14.626180 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1be158-e779-4e9b-ade1-a93ff29b3981 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:48:14.646146 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:48:14.690161 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:48:14.690834 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:48:14.691108 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:48:14.691375 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] PATH_INFO: `/v3/users/7595fe53d9bb416c8054fcedf96fc32f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:48:14.692520 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7595fe53d9bb416c8054fcedf96fc32f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:48:14.694492 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:48:14.748137 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7595fe53d9bb416c8054fcedf96fc32f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.756718 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7595fe53d9bb416c8054fcedf96fc32f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.757374 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7595fe53d9bb416c8054fcedf96fc32f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.769963 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7595fe53d9bb416c8054fcedf96fc32f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.770682 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7595fe53d9bb416c8054fcedf96fc32f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:14.790151 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1481/2931] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Aug 31 04:48:14 2022] DELETE /identity/v3/users/7595fe53d9bb416c8054fcedf96fc32f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:48:14.975943 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:48:14.978276 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:48:14.991926 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f040-c87b-4311-be3b-3603c0c4266c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:48:15.057354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13d95fead9ed4b918730e2198000e5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '529acf6d876d441682b1d4508345bc61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:48:15.057989 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:48:15.058324 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:48:15.058592 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] PATH_INFO: `/v3/projects/0c6b37ba935a4a1f9a7795d0f873a2da` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:48:15.059814 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6b37ba935a4a1f9a7795d0f873a2da)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:48:15.061770 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:48:15.087354 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:48:15.115538 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6b37ba935a4a1f9a7795d0f873a2da', 'request_id': 'req-2768df32-c0df-4d3c-8162-407a8427fc25'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:15.121731 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6b37ba935a4a1f9a7795d0f873a2da', 'request_id': 'req-2768df32-c0df-4d3c-8162-407a8427fc25'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:15.131188 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6b37ba935a4a1f9a7795d0f873a2da', 'request_id': 'req-2768df32-c0df-4d3c-8162-407a8427fc25'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:15.142318 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] The token cache is being invalidate because project 0c6b37ba935a4a1f9a7795d0f873a2da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:48:15.142616 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:15.143304 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2768df32-c0df-4d3c-8162-407a8427fc25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:48:15.145023 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1451/2932] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Aug 31 04:48:14 2022] DELETE /identity/v3/projects/0c6b37ba935a4a1f9a7795d0f873a2da => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:48:24.784009 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:48:25.139584 dv-cinder-855307-1-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:53:08.979612 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:53:08.981199 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:53:09.128040 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364845d-049f-4e38-93f4-a0a87626f8c2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:53:09.193563 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c13fd2-b86a-4159-9afb-257ab9a61de4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463279871f78480d9fee052518539b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c15c751e94034d7591fabc8910a7076b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:53:09.194179 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c13fd2-b86a-4159-9afb-257ab9a61de4 service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:53:09.194511 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c13fd2-b86a-4159-9afb-257ab9a61de4 service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:53:09.194794 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c13fd2-b86a-4159-9afb-257ab9a61de4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:53:09.196852 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-18c13fd2-b86a-4159-9afb-257ab9a61de4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:53:09.208688 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c13fd2-b86a-4159-9afb-257ab9a61de4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:53:09.210271 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c13fd2-b86a-4159-9afb-257ab9a61de4 service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:53:09.220526 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1482/2933] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Aug 31 04:53:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:53:19.216342 dv-cinder-855307-1-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}